{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "10_Object_Oriented_ML",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "GDgMuZNj7ERp",
        "AKQ1bVDQMtmC",
        "1DMwuOhwPCk7",
        "nA9FrU9LP2xE",
        "Ast9gfliULvU",
        "CqUbp3-MUrbp",
        "8aek-rFOVrEw",
        "CFLBjYseXDFK",
        "r9HarbSza9QM",
        "U7ZUTZbyeW8T"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bOChJSNXtC9g",
        "colab_type": "text"
      },
      "source": [
        "# Object Oriented ML"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OLIxEDq6VhvZ",
        "colab_type": "text"
      },
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/logo.png\" width=150>\n",
        "\n",
        "In this notebook, we will learn how to properly create and use classes & functions to solve ML tasks with PyTorch. We will be following this implementation structure in subsequent notebooks.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VoMq0eFRvugb",
        "colab_type": "text"
      },
      "source": [
        "# Overview"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qWro5T5qTJJL",
        "colab_type": "text"
      },
      "source": [
        "Here is the overview of the different classes and the functionality they will have."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f88fhoHFLIKg",
        "colab_type": "text"
      },
      "source": [
        "*   **Vocabulary**: dictionaries to convert between raw inputs and numerical form. Usually useful for NLP tasks to build and store vocabulary dictionaries.\n",
        "*   **Vectorizer**: instances of the vocabulary class for inputs and outputs and vectorizes the data for the model.\n",
        "*   **Dataset**: vectorizers to process the data and data splits.\n",
        "*   **Model**: PyTorch model to process inputs and return predictions.\n",
        "* **Trainer**: training operations using the dataset, vectorizer and model.\n",
        "* **Inference**: inference operations for new data\n",
        "\n",
        "Credit for the code structure in the implementations below and in subsequent lessons goes to these [contributors](https://github.com/joosthub/PyTorchNLPBook/graphs/contributors)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ODSOZctb0VQg",
        "colab_type": "text"
      },
      "source": [
        "# Configuration"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g2akVr26R--d",
        "colab_type": "text"
      },
      "source": [
        "We will be using these configurations throughout the notebook for our task of predicting nationality given a surname."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F4pjjYBUFgsI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "config = {\n",
        "  \"seed\": 1234,\n",
        "  \"cuda\": True,\n",
        "  \"data_url\": \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/surnames.csv\",\n",
        "  \"data_file\": \"datasets/surnames.csv\",\n",
        "  \"shuffle\": True,\n",
        "  \"train_size\": 0.7,\n",
        "  \"val_size\": 0.15,\n",
        "  \"test_size\": 0.15,\n",
        "  \"vectorizer_file\": \"vectorizer.json\",\n",
        "  \"model_file\": \"model.pth\",\n",
        "  \"save_dir\": \"experiments\",\n",
        "  \"num_epochs\": 20,\n",
        "  \"early_stopping_criteria\": 5,\n",
        "  \"learning_rate\": 1e-3,\n",
        "  \"batch_size\": 128,\n",
        "  \"fc\": {\n",
        "    \"hidden_dim\": 300,\n",
        "    \"dropout_p\": 0.1\n",
        "  }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xVi5AQCV2I_M",
        "colab_type": "text"
      },
      "source": [
        "# Set up"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MVZrmryoR4bq",
        "colab_type": "text"
      },
      "source": [
        "We're going to get set up for our task by setting reproducability seeds with NumPy and PyTorch. We will also create a unique directory to store our configurations, model, etc."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sF0YDJI4VM2a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Load PyTorch library\n",
        "!pip3 install torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F9z642sRVCAD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os\n",
        "import json\n",
        "import numpy as np\n",
        "import time\n",
        "import torch\n",
        "import uuid"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GDgMuZNj7ERp",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Qdd1FXOc5ujg",
        "colab": {}
      },
      "source": [
        "def set_seeds(seed, cuda):\n",
        "    \"\"\" Set Numpy and PyTorch seeds.\n",
        "    \"\"\"\n",
        "    np.random.seed(seed)\n",
        "    torch.manual_seed(seed)\n",
        "    if cuda:\n",
        "        torch.cuda.manual_seed_all(seed)\n",
        "    print (\"==> 🌱 Set NumPy and PyTorch seeds.\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VUxuU5aR7Yo4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generate_unique_id():\n",
        "    \"\"\"Generate a unique uuid\n",
        "    preceded by a epochtime.\n",
        "    \"\"\"\n",
        "    timestamp = int(time.time())\n",
        "    unique_id = \"{}_{}\".format(timestamp, uuid.uuid1())\n",
        "    print (\"==> 🔑 Generated unique id: {0}\".format(unique_id))\n",
        "    return unique_id"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "26PW-4qj9Vd1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def create_dirs(dirpath):\n",
        "    \"\"\"Creating directories.\n",
        "    \"\"\"\n",
        "    if not os.path.exists(dirpath):\n",
        "        os.makedirs(dirpath)\n",
        "        print (\"==> 📂 Created {0}\".format(dirpath))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_9gGo840BWSn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def check_cuda(cuda):\n",
        "    \"\"\"Check to see if GPU is available.\n",
        "    \"\"\"\n",
        "    if not torch.cuda.is_available():\n",
        "        cuda = False\n",
        "    device = torch.device(\"cuda\" if cuda else \"cpu\")\n",
        "    print (\"==> 💻 Device: {0}\".format(device))\n",
        "    return device"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uJ8rb_h97HYO",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "ec541b46-a019-40c2-9f5c-01112f460251",
        "id": "mVbPzN3A5uUa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Set seeds for reproducability\n",
        "set_seeds(seed=config[\"seed\"], cuda=config[\"cuda\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 🌱 Set NumPy and PyTorch seeds.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dMHZbx9tFg50",
        "colab_type": "code",
        "outputId": "cc4e9799-16a9-45d3-911c-7309cf11fab4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Generate unique experiment ID\n",
        "config[\"experiment_id\"] = generate_unique_id()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 🔑 Generated unique id: 1546545495_e864de5c-0f91-11e9-9542-0242ac1c0002\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GAmBykuO9iQW",
        "colab_type": "code",
        "outputId": "3fb2d4c3-4ccd-4741-b776-d3c02c21a9ef",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Create experiment directory\n",
        "config[\"save_dir\"] = os.path.join(config[\"save_dir\"], config[\"experiment_id\"])\n",
        "create_dirs(dirpath=config[\"save_dir\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 📂 Created experiments/1546545495_e864de5c-0f91-11e9-9542-0242ac1c0002\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k3jYbBbNFg3_",
        "colab_type": "code",
        "outputId": "3009ec2f-389d-4489-f344-ffd92bf26325",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Expand file paths to store components later\n",
        "config[\"vectorizer_file\"] = os.path.join(config[\"save_dir\"], config[\"vectorizer_file\"])\n",
        "config[\"model_file\"] = os.path.join(config[\"save_dir\"], config[\"model_file\"])\n",
        "print (\"Expanded filepaths: \")\n",
        "print (\"{}\".format(config[\"vectorizer_file\"]))\n",
        "print (\"{}\".format(config[\"model_file\"]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Expanded filepaths: \n",
            "experiments/1546545495_e864de5c-0f91-11e9-9542-0242ac1c0002/vectorizer.json\n",
            "experiments/1546545495_e864de5c-0f91-11e9-9542-0242ac1c0002/model.pth\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CywPpqLfTRoM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Save config\n",
        "config_fp = os.path.join(config[\"save_dir\"], \"config.json\")\n",
        "with open(config_fp, \"w\") as fp:\n",
        "    json.dump(config, fp)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-4PdHMemFgx2",
        "colab_type": "code",
        "outputId": "b1212a71-56d2-46c4-f682-2f86ff5464ab",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Check CUDA\n",
        "config[\"device\"] = check_cuda(cuda=config[\"cuda\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 💻 Device: cuda\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2H6Ty3RwMrGK",
        "colab_type": "text"
      },
      "source": [
        "# Load data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kW71YsE7Tcvc",
        "colab_type": "text"
      },
      "source": [
        "Get the data from the GitHub URL and then load it into a Pandas DataFrame."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vi66jJhKVTSE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import urllib"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AKQ1bVDQMtmC",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fRq8Q6exNS48",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def get_data(data_url, data_file):\n",
        "    \"\"\"Get data from GitHub to notebook's\n",
        "    local drive.\n",
        "    \"\"\"\n",
        "    # Create dataset directory\n",
        "    dirpath = os.path.dirname(data_file)\n",
        "    create_dirs(dirpath)\n",
        "    \n",
        "    # Fetch data\n",
        "    response = urllib.request.urlopen(config[\"data_url\"])\n",
        "    html = response.read()\n",
        "    with open(config[\"data_file\"], 'wb') as fp:\n",
        "        fp.write(html)\n",
        "    print (\"==> 🐙 Downloading data from GitHub to {0}\".format(data_file))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YVdiWWXeFgv2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def load_data(data_file):\n",
        "    \"\"\"Load data from CSV to Pandas DataFrame.\n",
        "    \"\"\"\n",
        "    # Load into DataFrame\n",
        "    df = pd.read_csv(data_file, header=0)\n",
        "    print (\"==> 🍣 Raw data:\")\n",
        "    print (df.head())\n",
        "    return df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gySOcLTGMvfW",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tez_Cw6D4nNt",
        "colab_type": "code",
        "outputId": "b979febc-22ee-4b66-caf9-f26f7fb65cc4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "# Get data from GitHub\n",
        "get_data(data_url=config[\"data_url\"], data_file=config[\"data_file\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 📂 Created datasets\n",
            "==> 🐙 Downloading data from GitHub to datasets/surnames.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1aBxFopR4nVg",
        "colab_type": "code",
        "outputId": "e96595d5-278e-4a56-8c33-eb33a14a6b9a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Load data into Pandas DataFrame\n",
        "df = load_data(data_file=config[\"data_file\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 🍣 Raw data:\n",
            "    surname nationality\n",
            "0  Woodford     English\n",
            "1      Coté      French\n",
            "2      Kore     English\n",
            "3     Koury      Arabic\n",
            "4    Lebzak     Russian\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4VRVh0lZO5JU",
        "colab_type": "text"
      },
      "source": [
        "# Split data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AS4i8TvwThtK",
        "colab_type": "text"
      },
      "source": [
        "Split the data into train, validation and test sets where each split has similar class distributions. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yrF4j1nPVdUR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import collections"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1DMwuOhwPCk7",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DKcxEise4nfO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def split_data(df, shuffle, train_size, val_size, test_size):\n",
        "    \"\"\"Split the data into train/val/test splits.\n",
        "    \"\"\"\n",
        "    # Split by category\n",
        "    by_category = collections.defaultdict(list)\n",
        "    for _, row in df.iterrows():\n",
        "        by_category[row.nationality].append(row.to_dict())\n",
        "    print (\"\\n==> 🛍️ Categories:\")\n",
        "    for category in by_category:\n",
        "        print (\"{0}: {1}\".format(category, len(by_category[category])))\n",
        "\n",
        "    # Create split data\n",
        "    final_list = []\n",
        "    for _, item_list in sorted(by_category.items()):\n",
        "        if shuffle:\n",
        "            np.random.shuffle(item_list)\n",
        "        n = len(item_list)\n",
        "        n_train = int(train_size*n)\n",
        "        n_val = int(val_size*n)\n",
        "        n_test = int(test_size*n)\n",
        "\n",
        "      # Give data point a split attribute\n",
        "        for item in item_list[:n_train]:\n",
        "            item['split'] = 'train'\n",
        "        for item in item_list[n_train:n_train+n_val]:\n",
        "            item['split'] = 'val'\n",
        "        for item in item_list[n_train+n_val:]:\n",
        "            item['split'] = 'test'\n",
        "\n",
        "        # Add to final list\n",
        "        final_list.extend(item_list)\n",
        "\n",
        "    # df with split datasets\n",
        "    split_df = pd.DataFrame(final_list)\n",
        "    print (\"\\n==> 🖖 Splits:\")\n",
        "    print (split_df[\"split\"].value_counts())\n",
        "\n",
        "    return split_df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zpw_x_B-PDxS",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cq54l2pk4ncu",
        "colab_type": "code",
        "outputId": "c74ebe36-94db-4e57-b195-5005cc9edeeb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 459
        }
      },
      "source": [
        "# Split data\n",
        "split_df = split_data(\n",
        "    df=df, shuffle=config[\"shuffle\"],\n",
        "    train_size=config[\"train_size\"],\n",
        "    val_size=config[\"val_size\"],\n",
        "    test_size=config[\"test_size\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "==> 🛍️ Categories:\n",
            "English: 2972\n",
            "French: 229\n",
            "Arabic: 1603\n",
            "Russian: 2373\n",
            "Japanese: 775\n",
            "Chinese: 220\n",
            "Italian: 600\n",
            "Czech: 414\n",
            "Irish: 183\n",
            "German: 576\n",
            "Greek: 156\n",
            "Spanish: 258\n",
            "Polish: 120\n",
            "Dutch: 236\n",
            "Vietnamese: 58\n",
            "Korean: 77\n",
            "Portuguese: 55\n",
            "Scottish: 75\n",
            "\n",
            "==> 🖖 Splits:\n",
            "train    7680\n",
            "test     1660\n",
            "val      1640\n",
            "Name: split, dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3Yi9RbtOP06X",
        "colab_type": "text"
      },
      "source": [
        "# Preprocessing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jN-9ieA8TtTW",
        "colab_type": "text"
      },
      "source": [
        "Preprocess the data in the DataFrame."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sVy_sX2mVhua",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import re"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nA9FrU9LP2xE",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IHHr--K04na-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess_text(text):\n",
        "    \"\"\"Basic text preprocessing.\n",
        "    \"\"\"\n",
        "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
        "    text = text.replace('\\n', ' ')\n",
        "    text = re.sub(r\"[^a-zA-Z.!?_]+\", r\" \", text)\n",
        "    text = text.strip()\n",
        "    return text"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FnSaB4GGP-8L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess_data(df):\n",
        "    \"\"\" Preprocess the DataFrame.\n",
        "    \"\"\"\n",
        "    df.surname = df.surname.apply(preprocess_text)\n",
        "    print (\"\\n==> 🚿 Preprocessing:\")\n",
        "    print (df.head())\n",
        "    return df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m0RfghamP34u",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qLRNMdHg4nZK",
        "colab_type": "code",
        "outputId": "a440eac6-97bc-43b3-c3a6-42881294762a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Preprocessing\n",
        "preprocessed_df = preprocess_data(split_df)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "==> 🚿 Preprocessing:\n",
            "  nationality  split  surname\n",
            "0      Arabic  train  bishara\n",
            "1      Arabic  train    nahas\n",
            "2      Arabic  train   ghanem\n",
            "3      Arabic  train  tannous\n",
            "4      Arabic  train  mikhail\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hPtdWOHGQXwQ",
        "colab_type": "text"
      },
      "source": [
        "# Vocabulary"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_oivqpaxTwy_",
        "colab_type": "text"
      },
      "source": [
        "Create vocabularies for the surnames and nationality classes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ast9gfliULvU",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5eXfXlTP5pI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Vocabulary(object):\n",
        "    def __init__(self, token_to_idx=None, add_unk=True, unk_token=\"<UNK>\"):\n",
        "\n",
        "        # Token to index\n",
        "        if token_to_idx is None:\n",
        "            token_to_idx = {}\n",
        "        self.token_to_idx = token_to_idx\n",
        "\n",
        "        # Index to token\n",
        "        self.idx_to_token = {idx: token \\\n",
        "                             for token, idx in self.token_to_idx.items()}\n",
        "        \n",
        "        # Add unknown token\n",
        "        self.add_unk = add_unk\n",
        "        self.unk_token = unk_token\n",
        "        if self.add_unk:\n",
        "            self.unk_index = self.add_token(self.unk_token)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'token_to_idx': self.token_to_idx,\n",
        "                'add_unk': self.add_unk, 'unk_token': self.unk_token}\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        return cls(**contents)\n",
        "\n",
        "    def add_token(self, token):\n",
        "        if token in self.token_to_idx:\n",
        "            index = self.token_to_idx[token]\n",
        "        else:\n",
        "            index = len(self.token_to_idx)\n",
        "            self.token_to_idx[token] = index\n",
        "            self.idx_to_token[index] = token\n",
        "        return index\n",
        "\n",
        "    def add_tokens(self, tokens):\n",
        "        return [self.add_token[token] for token in tokens]\n",
        "\n",
        "    def lookup_token(self, token):\n",
        "        if self.add_unk:\n",
        "            index = self.token_to_idx.get(token, self.unk_index)\n",
        "        else:\n",
        "            index =  self.token_to_idx[token]\n",
        "        return index\n",
        "\n",
        "    def lookup_index(self, index):\n",
        "        if index not in self.idx_to_token:\n",
        "            raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n",
        "        return self.idx_to_token[index]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Vocabulary(size=%d)>\" % len(self)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.token_to_idx)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7av4XJw9UOZO",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aFI3w39WP5to",
        "colab_type": "code",
        "outputId": "f3ca1663-355a-40c2-93da-f7fedf0e8711",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Vocabulary instance\n",
        "nationality_vocab = Vocabulary(add_unk=False)\n",
        "for index, row in preprocessed_df.iterrows():\n",
        "    nationality_vocab.add_token(row.nationality)\n",
        "print (nationality_vocab) # __str__\n",
        "print (len(nationality_vocab)) # __len__\n",
        "index = nationality_vocab.lookup_token(\"English\")\n",
        "print (index)\n",
        "print (nationality_vocab.lookup_index(index))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=18)>\n",
            "18\n",
            "4\n",
            "English\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wt6sCsBu238H",
        "colab_type": "text"
      },
      "source": [
        "# Vectorizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2F-_ZVD0Ul_h",
        "colab_type": "text"
      },
      "source": [
        "Vectorizes our data into numerical form using the vocabularies."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CqUbp3-MUrbp",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lFSaEs4L2TkY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SurnameVectorizer(object):\n",
        "    def __init__(self, surname_vocab, nationality_vocab):\n",
        "        self.surname_vocab = surname_vocab\n",
        "        self.nationality_vocab = nationality_vocab\n",
        "\n",
        "    def vectorize(self, surname):\n",
        "        one_hot = np.zeros(len(self.surname_vocab), dtype=np.float32)\n",
        "        for token in surname:\n",
        "            one_hot[self.surname_vocab.lookup_token(token)] = 1\n",
        "        return one_hot\n",
        "\n",
        "    def unvectorize(self, one_hot):\n",
        "        surname = [vectorizer.surname_vocab.lookup_index(index) \\\n",
        "            for index in np.where(one_hot==1)[0]]\n",
        "        return surname\n",
        "        \n",
        "    @classmethod\n",
        "    def from_dataframe(cls, df):\n",
        "        surname_vocab = Vocabulary(add_unk=True)\n",
        "        nationality_vocab = Vocabulary(add_unk=False)\n",
        "\n",
        "        # Create vocabularies\n",
        "        for index, row in df.iterrows():\n",
        "            for letter in row.surname: # char-level tokenization\n",
        "                surname_vocab.add_token(letter)\n",
        "            nationality_vocab.add_token(row.nationality)\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        surname_vocab = Vocabulary.from_serializable(contents['surname_vocab'])\n",
        "        nationality_vocab =  Vocabulary.from_serializable(contents['nationality_vocab'])\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'surname_vocab': self.surname_vocab.to_serializable(),\n",
        "                'nationality_vocab': self.nationality_vocab.to_serializable()}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UzZDR-ehUt3x",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IBf2a0nz4Ji_",
        "colab_type": "code",
        "outputId": "dae5cba1-e84e-40df-8b9f-5ab43305283d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Vectorizer instance\n",
        "vectorizer = SurnameVectorizer.from_dataframe(preprocessed_df)\n",
        "print (vectorizer.surname_vocab)\n",
        "print (vectorizer.nationality_vocab)\n",
        "one_hot = vectorizer.vectorize(preprocess_text(\"goku\"))\n",
        "print (one_hot)\n",
        "print (vectorizer.unvectorize(one_hot))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=28)>\n",
            "<Vocabulary(size=18)>\n",
            "[0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            " 0. 0. 0. 0.]\n",
            "['g', 'o', 'u', 'k']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N0QYigLGZJ6G",
        "colab_type": "text"
      },
      "source": [
        "**Note**: When we vectorize our input with bagged one hot encoded format, we lose all the structure in our name. This is a major disadvantage of representing our text in bagged one-hot encoded form but we will explore more semantic structure preserving encoding methods later."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vT7q4sh558yh",
        "colab_type": "text"
      },
      "source": [
        "# Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zaYqIwyUVkOq",
        "colab_type": "text"
      },
      "source": [
        "The Dataset will create vectorized data from the preprocessed data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4fYxY_Eq-Tso",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import random\n",
        "from torch.utils.data import Dataset, DataLoader"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8aek-rFOVrEw",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XFQf4ikx5pp1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SurnameDataset(Dataset):\n",
        "    def __init__(self, df, vectorizer, infer=False):\n",
        "        self.df = df\n",
        "        self.vectorizer = vectorizer\n",
        "        \n",
        "        # Data splits\n",
        "        if not infer:\n",
        "            self.train_df = self.df[self.df.split=='train']\n",
        "            self.train_size = len(self.train_df)\n",
        "            self.val_df = self.df[self.df.split=='val']\n",
        "            self.val_size = len(self.val_df)\n",
        "            self.test_df = self.df[self.df.split=='test']\n",
        "            self.test_size = len(self.test_df)\n",
        "            self.lookup_dict = {'train': (self.train_df, self.train_size), \n",
        "                                'val': (self.val_df, self.val_size),\n",
        "                                'test': (self.test_df, self.test_size)}\n",
        "            self.set_split('train')\n",
        "\n",
        "            # Class weights (for imbalances)\n",
        "            class_counts = df.nationality.value_counts().to_dict()\n",
        "            def sort_key(item):\n",
        "                return self.vectorizer.nationality_vocab.lookup_token(item[0])\n",
        "            sorted_counts = sorted(class_counts.items(), key=sort_key)\n",
        "            frequencies = [count for _, count in sorted_counts]\n",
        "            self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n",
        "        elif infer:\n",
        "            self.infer_df = self.df[self.df.split==\"infer\"]\n",
        "            self.infer_size = len(self.infer_df)\n",
        "            self.lookup_dict = {'infer': (self.infer_df, self.infer_size)}\n",
        "            self.set_split('infer')\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_make_vectorizer(cls, df):\n",
        "        train_df = df[df.split=='train']\n",
        "        return cls(df, SurnameVectorizer.from_dataframe(train_df))\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_load_vectorizer(cls, df, vectorizer_filepath):\n",
        "        vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n",
        "        return cls(df, vectorizer)\n",
        "\n",
        "    def load_vectorizer_only(vectorizer_filepath):\n",
        "        with open(vectorizer_filepath) as fp:\n",
        "            return SurnameVectorizer.from_serializable(json.load(fp))\n",
        "\n",
        "    def save_vectorizer(self, vectorizer_filepath):\n",
        "        with open(vectorizer_filepath, \"w\") as fp:\n",
        "            json.dump(self.vectorizer.to_serializable(), fp)\n",
        "\n",
        "    def set_split(self, split=\"train\"):\n",
        "        self.target_split = split\n",
        "        self.target_df, self.target_size = self.lookup_dict[split]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Dataset(split={0}, size={1})>\".format(\n",
        "            self.target_split, self.target_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.target_size\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        row = self.target_df.iloc[index]\n",
        "        surname_vector = self.vectorizer.vectorize(row.surname)\n",
        "        nationality_index = self.vectorizer.nationality_vocab.lookup_token(row.nationality)\n",
        "        return {'surname': surname_vector, 'nationality': nationality_index}\n",
        "\n",
        "    def get_num_batches(self, batch_size):\n",
        "        return len(self) // batch_size\n",
        "\n",
        "    def generate_batches(self, batch_size, shuffle=True, drop_last=False, device=\"cpu\"):\n",
        "        dataloader = DataLoader(dataset=self, batch_size=batch_size, \n",
        "                                shuffle=shuffle, drop_last=drop_last)\n",
        "        for data_dict in dataloader:\n",
        "            out_data_dict = {}\n",
        "            for name, tensor in data_dict.items():\n",
        "                out_data_dict[name] = data_dict[name].to(device)\n",
        "            yield out_data_dict"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IkozejHzV51b",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def sample(dataset):\n",
        "    \"\"\"Some sanity checks on the dataset.\n",
        "    \"\"\"\n",
        "    sample_idx = random.randint(0,len(dataset))\n",
        "    sample = dataset[sample_idx]\n",
        "    print (\"\\n==> 🔢 Dataset:\")\n",
        "    print (\"Random sample: {0}\".format(sample))\n",
        "    print (\"Unvectorized surname: {0}\".format(\n",
        "        dataset.vectorizer.unvectorize(sample['surname'])))\n",
        "    print (\"Unvectorized nationality: {0}\".format(\n",
        "        dataset.vectorizer.nationality_vocab.lookup_index(sample['nationality'])))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KjjsaFnMVt3y",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QhAJn2H3-vvu",
        "colab_type": "code",
        "outputId": "0cf90db7-b812-45f1-f578-0b4185cd6b74",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "# Load dataset and vectorizer\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(preprocessed_df)\n",
        "dataset.save_vectorizer(config[\"vectorizer_file\"])\n",
        "vectorizer = dataset.vectorizer\n",
        "print (dataset.class_weights)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([0.0006, 0.0045, 0.0024, 0.0042, 0.0003, 0.0044, 0.0017, 0.0064, 0.0055,\n",
            "        0.0017, 0.0013, 0.0130, 0.0083, 0.0182, 0.0004, 0.0133, 0.0039, 0.0172])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jt_tb9ZqWRrt",
        "colab_type": "code",
        "outputId": "84c75171-23de-4b4d-856c-e2a91cfec9c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "# Sample checks\n",
        "sample(dataset=dataset)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "==> 🔢 Dataset:\n",
            "Random sample: {'surname': array([0., 1., 1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
            "       1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32), 'nationality': 0}\n",
            "Unvectorized surname: ['b', 'i', 'a', 'z']\n",
            "Unvectorized nationality: Arabic\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q8CAcVWRCVtm",
        "colab_type": "text"
      },
      "source": [
        "# Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3JZ9whVlXBaZ",
        "colab_type": "text"
      },
      "source": [
        "Basic MLP architecture for surname classification."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K4yDxHIe_hGv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CFLBjYseXDFK",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_bbJqIPRCbuZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SurnameModel(nn.Module):\n",
        "    def __init__(self, input_dim, hidden_dim, output_dim, dropout_p):\n",
        "        super(SurnameModel, self).__init__()\n",
        "        self.fc1 = nn.Linear(input_dim, hidden_dim)\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "        self.fc2 = nn.Linear(hidden_dim, output_dim)\n",
        "\n",
        "    def forward(self, x_in, apply_softmax=False):\n",
        "        z = F.relu(self.fc1(x_in))\n",
        "        z = self.dropout(z)\n",
        "        y_pred = self.fc2(z)\n",
        "\n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mIN_9XWAXEfH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def initialize_model(config, vectorizer):\n",
        "    \"\"\"Initialize the model.\n",
        "    \"\"\"\n",
        "    print (\"\\n==> 🚀 Initializing model:\")\n",
        "    model = SurnameModel(\n",
        "        input_dim=len(vectorizer.surname_vocab), \n",
        "        hidden_dim=config[\"fc\"][\"hidden_dim\"], \n",
        "        output_dim=len(vectorizer.nationality_vocab),\n",
        "        dropout_p=config[\"fc\"][\"dropout_p\"])\n",
        "    print (model.named_modules)\n",
        "    return model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pSAixX_PXElV",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "93KUSpSGXGNN",
        "colab_type": "code",
        "outputId": "0844aa10-94c9-4753-fcbb-654edde5cd3e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Initializing model\n",
        "model = initialize_model(config=config, vectorizer=vectorizer)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "==> 🚀 Initializing model:\n",
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (fc1): Linear(in_features=28, out_features=300, bias=True)\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc2): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p0Hr9OohDmPI",
        "colab_type": "text"
      },
      "source": [
        "# Training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lQOmETa5X8XW",
        "colab_type": "text"
      },
      "source": [
        "Training operations for surname classification."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UWYC2MfiKh8o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import torch.optim as optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9HarbSza9QM",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8qbr6A3yrgaU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def compute_accuracy(y_pred, y_target):\n",
        "    _, y_pred_indices = y_pred.max(dim=1)\n",
        "    n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n",
        "    return n_correct / len(y_pred_indices) * 100"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1hrrkLS2ra0K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def update_train_state(model, train_state):\n",
        "    \"\"\" Update train state during training.\n",
        "    \"\"\"\n",
        "    # Verbose\n",
        "    print (\"[EPOCH]: {0} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n",
        "      train_state['epoch_index'], train_state['learning_rate'], \n",
        "        train_state['train_loss'][-1], train_state['train_acc'][-1], \n",
        "        train_state['val_loss'][-1], train_state['val_acc'][-1]))\n",
        "\n",
        "    # Save one model at least\n",
        "    if train_state['epoch_index'] == 0:\n",
        "        torch.save(model.state_dict(), train_state['model_filename'])\n",
        "        train_state['stop_early'] = False\n",
        "\n",
        "    # Save model if performance improved\n",
        "    elif train_state['epoch_index'] >= 1:\n",
        "        loss_tm1, loss_t = train_state['val_loss'][-2:]\n",
        "\n",
        "        # If loss worsened\n",
        "        if loss_t >= train_state['early_stopping_best_val']:\n",
        "            # Update step\n",
        "            train_state['early_stopping_step'] += 1\n",
        "\n",
        "        # Loss decreased\n",
        "        else:\n",
        "            # Save the best model\n",
        "            if loss_t < train_state['early_stopping_best_val']:\n",
        "                torch.save(model.state_dict(), train_state['model_filename'])\n",
        "\n",
        "            # Reset early stopping step\n",
        "            train_state['early_stopping_step'] = 0\n",
        "\n",
        "        # Stop early ?\n",
        "        train_state['stop_early'] = train_state['early_stopping_step'] \\\n",
        "          >= train_state['early_stopping_criteria']\n",
        "    return train_state"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dAux5rEXsjPj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Trainer(object):\n",
        "    def __init__(self, dataset, model, model_file, device, shuffle, \n",
        "               num_epochs, batch_size, learning_rate, early_stopping_criteria):\n",
        "        self.dataset = dataset\n",
        "        self.class_weights = dataset.class_weights.to(device)\n",
        "        self.model = model.to(device)\n",
        "        self.device = device\n",
        "        self.shuffle = shuffle\n",
        "        self.num_epochs = num_epochs\n",
        "        self.batch_size = batch_size\n",
        "        self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n",
        "        self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n",
        "        self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n",
        "            optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n",
        "        self.train_state = {\n",
        "            'done_training': False,\n",
        "            'stop_early': False, \n",
        "            'early_stopping_step': 0,\n",
        "            'early_stopping_best_val': 1e8,\n",
        "            'early_stopping_criteria': early_stopping_criteria,\n",
        "            'learning_rate': learning_rate,\n",
        "            'epoch_index': 0,\n",
        "            'train_loss': [],\n",
        "            'train_acc': [],\n",
        "            'val_loss': [],\n",
        "            'val_acc': [],\n",
        "            'test_loss': -1,\n",
        "            'test_acc': -1,\n",
        "            'model_filename': model_file}\n",
        "  \n",
        "    def run_train_loop(self):\n",
        "        print (\"==> 🏋 Training:\")\n",
        "        for epoch_index in range(self.num_epochs):\n",
        "            self.train_state['epoch_index'] = epoch_index\n",
        "      \n",
        "            # Iterate over train dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0, set train mode on\n",
        "            self.dataset.set_split('train')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, shuffle=self.shuffle, \n",
        "                device=self.device)\n",
        "            running_loss = 0.0\n",
        "            running_acc = 0.0\n",
        "            self.model.train()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "                # zero the gradients\n",
        "                self.optimizer.zero_grad()\n",
        "\n",
        "                # compute the output\n",
        "                y_pred = self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute gradients using loss\n",
        "                loss.backward()\n",
        "\n",
        "                # use optimizer to take a gradient step\n",
        "                self.optimizer.step()\n",
        "                \n",
        "                # compute the accuracy\n",
        "                acc_t = compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['train_loss'].append(running_loss)\n",
        "            self.train_state['train_acc'].append(running_acc)\n",
        "\n",
        "            # Iterate over val dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "            self.dataset.set_split('val')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.\n",
        "            running_acc = 0.\n",
        "            self.model.eval()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "\n",
        "                # compute the output\n",
        "                y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.to(\"cpu\").item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute the accuracy\n",
        "                acc_t = compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['val_loss'].append(running_loss)\n",
        "            self.train_state['val_acc'].append(running_acc)\n",
        "\n",
        "            self.train_state = update_train_state(model=self.model, train_state=self.train_state)\n",
        "            self.scheduler.step(self.train_state['val_loss'][-1])\n",
        "            if self.train_state['stop_early']:\n",
        "                break\n",
        "          \n",
        "    def run_test_loop(self):\n",
        "        # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "        self.dataset.set_split('test')\n",
        "        batch_generator = self.dataset.generate_batches(\n",
        "            batch_size=self.batch_size, shuffle=self.shuffle, device=self.device)\n",
        "        running_loss = 0.0\n",
        "        running_acc = 0.0\n",
        "        self.model.eval()\n",
        "\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "            # compute the loss\n",
        "            loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "            loss_t = loss.item()\n",
        "            running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "            # compute the accuracy\n",
        "            acc_t = compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "            running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "        self.train_state['test_loss'] = running_loss\n",
        "        self.train_state['test_acc'] = running_acc\n",
        "        \n",
        "        # Verbose\n",
        "        print (\"==> 💯 Test performance:\")\n",
        "        print (\"Test loss: {0:.2f}\".format(self.train_state['test_loss']))\n",
        "        print (\"Test Accuracy: {0:.1f}%\".format(self.train_state['test_acc']))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XUCncj-Br2y1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def plot_performance(train_state, save_dir, show_plot=True):\n",
        "    \"\"\" Plot loss and accuracy.\n",
        "    \"\"\"\n",
        "    # Figure size\n",
        "    plt.figure(figsize=(15,5))\n",
        "\n",
        "    # Plot Loss\n",
        "    plt.subplot(1, 2, 1)\n",
        "    plt.title(\"Loss\")\n",
        "    plt.plot(train_state[\"train_loss\"], label=\"train\")\n",
        "    plt.plot(train_state[\"val_loss\"], label=\"val\")\n",
        "    plt.legend(loc='upper right')\n",
        "\n",
        "    # Plot Accuracy\n",
        "    plt.subplot(1, 2, 2)\n",
        "    plt.title(\"Accuracy\")\n",
        "    plt.plot(train_state[\"train_acc\"], label=\"train\")\n",
        "    plt.plot(train_state[\"val_acc\"], label=\"val\")\n",
        "    plt.legend(loc='lower right')\n",
        "\n",
        "    # Save figure\n",
        "    plt.savefig(os.path.join(save_dir, \"performance.png\"))\n",
        "\n",
        "    # Show plots\n",
        "    if show_plot:\n",
        "        print (\"==> 📈 Metric plots:\")\n",
        "        plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XubhezdeshJX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def save_train_state(train_state, save_dir):\n",
        "    train_state[\"done_training\"] = True\n",
        "    with open(os.path.join(save_dir, \"train_state.json\"), \"w\") as fp:\n",
        "        json.dump(train_state, fp)\n",
        "    print (\"==> ✅ Training complete!\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ypH3nwWTbGgQ",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-5RrfYBpJFkg",
        "colab_type": "code",
        "outputId": "13961f94-3481-470a-a38b-e33fe1ceb3f2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 374
        }
      },
      "source": [
        "# Training\n",
        "trainer = Trainer(\n",
        "    dataset=dataset, model=model, model_file=config[\"model_file\"],\n",
        "    device=config[\"device\"], shuffle=config[\"shuffle\"], \n",
        "    num_epochs=config[\"num_epochs\"], batch_size=config[\"batch_size\"], \n",
        "    learning_rate=config[\"learning_rate\"], \n",
        "    early_stopping_criteria=config[\"early_stopping_criteria\"])\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 🏋 Training:\n",
            "[EPOCH]: 0 | [LR]: 0.001 | [TRAIN LOSS]: 2.80 | [TRAIN ACC]: 24.2% | [VAL LOSS]: 2.67 | [VAL ACC]: 38.6%\n",
            "[EPOCH]: 1 | [LR]: 0.001 | [TRAIN LOSS]: 2.52 | [TRAIN ACC]: 39.4% | [VAL LOSS]: 2.42 | [VAL ACC]: 34.8%\n",
            "[EPOCH]: 2 | [LR]: 0.001 | [TRAIN LOSS]: 2.27 | [TRAIN ACC]: 37.6% | [VAL LOSS]: 2.25 | [VAL ACC]: 38.0%\n",
            "[EPOCH]: 3 | [LR]: 0.001 | [TRAIN LOSS]: 2.12 | [TRAIN ACC]: 38.2% | [VAL LOSS]: 2.16 | [VAL ACC]: 38.7%\n",
            "[EPOCH]: 4 | [LR]: 0.001 | [TRAIN LOSS]: 2.02 | [TRAIN ACC]: 38.0% | [VAL LOSS]: 2.11 | [VAL ACC]: 36.8%\n",
            "[EPOCH]: 5 | [LR]: 0.001 | [TRAIN LOSS]: 1.97 | [TRAIN ACC]: 38.1% | [VAL LOSS]: 2.06 | [VAL ACC]: 37.1%\n",
            "[EPOCH]: 6 | [LR]: 0.001 | [TRAIN LOSS]: 1.91 | [TRAIN ACC]: 39.2% | [VAL LOSS]: 2.02 | [VAL ACC]: 37.7%\n",
            "[EPOCH]: 7 | [LR]: 0.001 | [TRAIN LOSS]: 1.87 | [TRAIN ACC]: 39.2% | [VAL LOSS]: 2.01 | [VAL ACC]: 37.1%\n",
            "[EPOCH]: 8 | [LR]: 0.001 | [TRAIN LOSS]: 1.84 | [TRAIN ACC]: 39.0% | [VAL LOSS]: 1.99 | [VAL ACC]: 37.1%\n",
            "[EPOCH]: 9 | [LR]: 0.001 | [TRAIN LOSS]: 1.81 | [TRAIN ACC]: 40.0% | [VAL LOSS]: 1.96 | [VAL ACC]: 37.8%\n",
            "[EPOCH]: 10 | [LR]: 0.001 | [TRAIN LOSS]: 1.79 | [TRAIN ACC]: 40.4% | [VAL LOSS]: 1.95 | [VAL ACC]: 38.5%\n",
            "[EPOCH]: 11 | [LR]: 0.001 | [TRAIN LOSS]: 1.76 | [TRAIN ACC]: 40.6% | [VAL LOSS]: 1.94 | [VAL ACC]: 37.4%\n",
            "[EPOCH]: 12 | [LR]: 0.001 | [TRAIN LOSS]: 1.74 | [TRAIN ACC]: 40.6% | [VAL LOSS]: 1.93 | [VAL ACC]: 38.7%\n",
            "[EPOCH]: 13 | [LR]: 0.001 | [TRAIN LOSS]: 1.72 | [TRAIN ACC]: 41.8% | [VAL LOSS]: 1.93 | [VAL ACC]: 40.7%\n",
            "[EPOCH]: 14 | [LR]: 0.001 | [TRAIN LOSS]: 1.70 | [TRAIN ACC]: 41.2% | [VAL LOSS]: 1.93 | [VAL ACC]: 38.5%\n",
            "[EPOCH]: 15 | [LR]: 0.001 | [TRAIN LOSS]: 1.67 | [TRAIN ACC]: 42.4% | [VAL LOSS]: 1.89 | [VAL ACC]: 39.1%\n",
            "[EPOCH]: 16 | [LR]: 0.001 | [TRAIN LOSS]: 1.66 | [TRAIN ACC]: 42.5% | [VAL LOSS]: 1.89 | [VAL ACC]: 37.9%\n",
            "[EPOCH]: 17 | [LR]: 0.001 | [TRAIN LOSS]: 1.64 | [TRAIN ACC]: 43.0% | [VAL LOSS]: 1.89 | [VAL ACC]: 40.0%\n",
            "[EPOCH]: 18 | [LR]: 0.001 | [TRAIN LOSS]: 1.63 | [TRAIN ACC]: 43.2% | [VAL LOSS]: 1.89 | [VAL ACC]: 40.1%\n",
            "[EPOCH]: 19 | [LR]: 0.001 | [TRAIN LOSS]: 1.61 | [TRAIN ACC]: 44.1% | [VAL LOSS]: 1.88 | [VAL ACC]: 40.3%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fgfpWoNRN_wu",
        "colab_type": "code",
        "outputId": "e2bd29ea-99b4-4ca9-a290-15f15b771912",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 352
        }
      },
      "source": [
        "# Plot performance\n",
        "plot_performance(train_state=trainer.train_state, \n",
        "                 save_dir=config[\"save_dir\"], show_plot=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 📈 Metric plots:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVXX+x/HXXdn3VRBQkE1AUcF9\nxX2pNEttdbKZppqWqZmpqX7VrG0zTevMNG22mVkulFpmmuSSC6CIIMjiwg6XfYfLvff3B0YRLqjA\n5eLn+Xj44N5z7j3nfa8ofO73ez5fhclkMiGEEEIIIYQQwuyU5g4ghBBCCCGEEKKdFGhCCCGEEEII\n0U9IgSaEEEIIIYQQ/YQUaEIIIYQQQgjRT0iBJoQQQgghhBD9hBRoQgghhBBCCNFPSIEmxGUKDQ2l\npKTE3DGEEEKIPrFixQquvfZac8cQYsCTAk0IIYQQQlxQVlYWDg4O+Pj4cOTIEXPHEWJAkwJNiB7W\n0tLCU089xdy5c5k/fz7PPfccBoMBgI8++oj58+czb948brjhBrKzsy+4XQghhOgPNm3axLx581i0\naBHx8fEd2+Pj45k7dy5z587lD3/4A62trefdfvDgQWbPnt3x3J/ef+211/i///s/brjhBt577z2M\nRiN//vOfmTt3LnFxcfzhD39Ar9cDUFlZyd13383MmTO55ppr2Lt3LwkJCSxatKhT5uuvv54dO3b0\n9lsjRI9TmzuAEAPN+++/T0lJCVu3bqWtrY1bb72VLVu2MHPmTF555RV27dqFvb09X331FQkJCQwa\nNOic24ODg839UoQQQggMBgPffPMNv/nNb1CpVLz44ou0trZSVlbG888/T3x8PJ6entx///188MEH\nzJs375zbo6KiLnie7777js8//xxXV1e+/vprkpKS2LJlC0ajkSVLlvDll19y3XXX8eKLLxIUFMQb\nb7zB8ePHueOOO9izZw86nY7MzEzCwsIoKioiLy+PqVOn9tG7JETPkQJNiB6WkJDAqlWrUKvVqNVq\nrrnmGvbt28eCBQtQKBSsX7+eRYsWMX/+fAD0ev05twshhBD9wd69e4mKisLe3h6AsWPHsmvXLqqr\nqxk1ahReXl4AvPjii6hUKjZs2HDO7cnJyRc8z8iRI3F1dQVg7ty5zJgxA41GA0BUVBT5+flAeyH3\n1ltvATB8+HB27tyJVqtl7ty5bN26lbCwMHbs2MHMmTPRarU9/4YI0ctkiqMQPayyshInJ6eO+05O\nTlRUVKDRaHjvvfc4fPgwc+fO5eabb+bEiRPn3S6EEEL0Bxs3biQhIYGYmBhiYmLYvn07mzZtoqqq\nCkdHx47HWVlZoVarz7v9Yn76s7OyspJHH32UuXPnMm/ePHbu3InJZAKguroaBweHjsf+UDguXLiQ\nrVu3ArBjxw4WLFhwZS9cCDORAk2IHubu7k51dXXH/erqatzd3YH2T/peffVV9u/fz+TJk3n66acv\nuF0IIYQwp5qaGg4dOsTBgwdJSkoiKSmJxMREjh07hlKppKqqquOx9fX1lJeX4+Lics7tKpWq45ps\ngNra2vOe96WXXkKtVrN582a2bdvGtGnTOvY5Ozt3On5BQQF6vZ7Y2Fja2trYtWsX2dnZTJw4safe\nBiH6lBRoQvSw6dOns379egwGA42NjXz++edMmzaNEydO8MADD9Da2opWqyUyMhKFQnHe7UIIIYS5\nbd26lfHjx3eaKqhWq5k8eTKtra0cPnyYgoICTCYTTz/9NOvXr2fatGnn3O7h4YFOp6OiogKDwcDm\nzZvPe96KigpCQkLQarVkZmZy5MgRGhsbAYiLi2PTpk0A5OTkcP3112MwGFAqlSxYsIC//vWvxMXF\ndUyPFMLSyDVoQlyB2267DZVK1XH/b3/7G7fddhv5+fksXLgQhULBvHnzOq4rGzx4MIsWLUKj0WBn\nZ8dTTz1FSEjIObcLIYQQ5hYfH8/KlSu7bJ89ezb/+c9/+Mtf/sLKlStRqVRERUVxxx13YGVldd7t\nS5cuZfHixfj4+HDdddeRkZFxzvOuWrWKRx99lI0bNxITE8Ojjz7KE088wYgRI/jDH/7Ao48+Slxc\nHHZ2dvzzn//E2toaaJ/muHr1apneKCyawvTDhF4hhBBCCCEsWHl5OUuWLCEhIaHTB6hCWBKZ4iiE\nEEIIIQaEV199lZtuukmKM2HRpEATQgghhBAWrby8nJkzZ1JeXs6qVavMHUeIKyJTHIUQQohuaG5u\nZtGiRdx7770cOnSI9PR0nJ2dAbjzzjuZPn16p8c/88wzHD16FIVCweOPP86IESPMkFoIIYSlkSYh\nQgghRDf897//7bRO08MPP8yMGTPO+dhDhw5x5swZ1q1bR25uLo8//jjr1q3rq6hCCCEsmExxFEII\nIS4iNzeXnJycLqNk57N//35mzZoFQFBQEDU1NdTX1/diQiGEEANFn4+g6XR1V3wMFxdbqqoaeyBN\n35LcfccSM4Nl5rbEzGCZuS0xs4eHg7kj9Ijnn3+eJ598kvj4+I5tH330EatXr8bNzY0nn3wSV1fX\njn3l5eVERER03Hd1dUWn02Fvb3/B88jPSMvKbYmZwTJzW2JmsMzclpgZLC/3hX4+WuQUR7XaMjvz\nSO6+Y4mZwTJzW2JmsMzclph5IIiPjyc6Oho/P7+Obddddx3Ozs6Eh4fz5ptv8vrrr19w/cLuXu7t\n4mLbI3/PlloYW2JuS8wMlpnbEjODZea2xMxgubl/ziILNCGEEKKvJCQkkJ+fT0JCAiUlJWi1Wv7y\nl78QHh4OQFxcHH/60586PcfT05Py8vKO+2VlZXh4eFz0XD3x6a+Hh0OPjMT1NUvMbYmZwTJzW2Jm\nsMzclpgZLC/3gBtBE0IIIfrKyy+/3HH7tddew9fXl7Vr1+Ln54efnx8HDx4kODi403MmTZrEa6+9\nxooVK0hPT8fT0/Oi0xuFEEIIkAJNCCGEuGS33HILv/3tb7GxscHW1pZnn30WgIceeohnn32W0aNH\nExERwYoVK1AoFDz99NNmTiyEEMJSSIEmhBBCdNP999/fcXvDhg1d9r/00ksdt3//+9/3SSYhhBAD\ni7TZF0IIIYQQQoh+Qgo0IYQQQgghhOgnujXF8YUXXiA5OZm2tjZ+/etfM2fOnI59a9as4YsvvkCp\nVBIZGckTTzzRa2GFEEIIIYQQYiC76AjagQMHyM7OZt26dbz99ts888wzHfvq6+t55513WLNmDWvX\nriU3N5eUlJReDSyEEOLyJSTs7NbjXnnlRYqKCns5jRBCCCF+7qIjaLGxsYwYMQIAR0dHmpqaMBgM\nqFQqNBoNGo2GxsZGbG1taWpqwsnJqddDCyGEuHTFxUXs2PE106fPvOhjH3zwd32QSAghhBA/d9EC\nTaVSYWtrC8D69euZOnUqKpUKACsrK37zm98wa9YsrKysWLhwIUOHDu3VwE0tbXyxJ5fRQW5YaVS9\nei4hhBhI/vWv58nISGfKlFjmzJlPcXERL7/8H5599i/odGU0NTWxatVdTJo0hfvuu4uHH36EXbt2\n0tBQT17eGQoLC3jggd8xYcIkc78UIYQQos8ZjSYOZZTi4WxDkG/vDUp1u83+jh07WL9+Pe+++27H\ntvr6ev73v/+xbds27O3tWblyJZmZmYSFhZ33OC4utqjVl19YHUwr5q34NH6xcDhL44Iv/oR+5kKr\nhvdnlpjbEjODZea2xMxgmbmvJPM99/yaNWvWEBwczMmTJ/nss3VUVFQwc+Z0lixZQn5+Pg8++CCL\nFy9Aq1Xj4mKHnZ0VRUV5vP/+anbv3s0nn3zCtdfO68FXJIQQQvR/ReUNvPdVJjmFNcSGeXKPuQu0\nPXv28MYbb/D222/j4PDjLwe5ubn4+fnh6uoKQExMDGlpaRcs0KqqGq8osIeDFqVSwZ4jBUyN8r6i\nY/U1Dw8HdLo6c8e4ZJaY2xIzg2XmtsTMYP7cn36bQ2Jm2SU9R6VSYDCYzrs/NsyTZXHDzru/urqR\nlhY9DQ0tBAaGoNPV0dam5NChZNas+RiFQklFRSU6XR2trW1UVTXQ0NBCaGgEOl0dVlYOVFZWX9L7\nZolFsBBCCPGDNoORrw7msXnfKdoMJmLDPLlldkivnvOiBVpdXR0vvPAC7733Hs7Ozp32+fr6kpub\nS3NzM9bW1qSlpTFt2rReCwvgYKslMtCN1JxyqupacHGw6tXzCSHEQKTRaAD45ptt1NbW8u9/v01t\nbS2//OVtXR77w7R2AJPp/AWiEEIIMZCcLqnl3a2ZFOjqcbLXctucUEaHePT6eS9aoH355ZdUVVXx\n29/+tmPbuHHjCA0NZfbs2dx5553cfvvtqFQqRo0aRUxMTK8GBpgYNYjUnHIOZ+mYOWZwr59PCCF6\n2rK4YRcc7TqXKx31UyqVGAyGTtuqq6sZNMgHpVLJd999i16vv+zjCyGEEANBq97A5/tO8fXBfIwm\nE1NGDGJ53DBsrTV9cv6LFmjLly9n+fLl592/YsUKVqxY0aOhLmZ81CDe2HRMCjQhhLgEAQFDOXEi\nk0GDfDpmREyfHscf//gwx4+nsXDhtXh6erJ69VtmTiqEEEKYx4m8Kt77KpPSqibcnaz5xfwwhg9x\n7dMM3W4S0p+4OdkQ5OPIibxq6hpbcbDVmjuSEEL0ey4uLmzcuLXTtkGDfHj//U867s+ZMx+AO+74\nFQCBgT+O8gUGDuP119/sg6RCCCFE32pqaWN9Qi67jhSiAGbH+HH91ECstH3fNd4iCzSA0aEe5BbV\nkpJTzpQRPuaOI4QQQgghhLBAqbnlfPD1CSprW/Bxt+OO+WG92kb/Yiy3QAvx4LNduRw+oZMCTQgh\nhBBCCHFJ6hpb+WRnNvvTS1EpFVw7aQgLJwxBo1aaNZfFFmheLrYM9rAn/XQlTS1t2FhZ7EsRQggh\nhBBC9BGTyURiZhlrvsmirlHPEG8H7lgQjp+nvbmjARZcoAGMCfXg8731HDtZwdhwL3PHEUIIIYQQ\nQvRjVXUtfPj1CVJyytGolSybMYzZsYNRKc07avZTll2ghXjw+d5TJJ/QSYEmhBBCCCGEOCeTycSe\n1GLWfZtDU0sboX7O/GJBGF4utuaO1oVFF2i+HnZ4utiQmluBvs2ARt33XVaEEEIIIYQQ5mEymWhu\nNVBW2UheSR31zXoamvQ0NLfR0KSnvklPQ7OeovIGThXXYa1VcfvcUKZG+6BUKMwd/5wsukBTKBSM\nCfHgq4N5pJ+qIjrY3dyRhBDC4t1wwzV88ME6bG3736eKQgghrh5VdS0cO1lBTUNre9F1tvDqKMLO\n3jcYTd063sggN26bG4qro3UvJ78yFl2gQXu7/a8O5pGcVSYFmhBCCCGEEBasVW/gSHY5+44Vk366\nEtM5ai+FAuysNdjZaPBwscHOWoObsw0qBdif3W5no/7xtrUaexsNttaavn9Bl8HiC7ShgxxxcbAi\nJbucNoMRtar/XOAnhBD9yapVt/DMMy/i7e1NSUkxjz32Ozw8PGlqaqK5uZmHHvoDw4dHmjumEEKI\nq4zJZCK3qJZ9x4o5lFFGU0sbAEG+jowf7o2Xa3sRZm+jwc5ag7WVqsv0RA8PB3S6OnPE73EWX6Ap\nFQpGB3uw83ABWfnVDB/iau5IQgjRL02dOoN9+3azdOky9uz5jqlTZxAUFMzUqdNJTk5kzZr3+fvf\n/2HumEIIIa4SlbXNfJ9Wwr5jxZRWNQHg4mBF3GhfJkZ6M8jNzswJzcPiCzRon+a483AByVk6KdCE\nEBZhY84WjpQdu6TnqJSKC86zH+UZxfXDFp13/9SpM3j99ZdZunQZe/d+x333PcQnn3zI2rUfotfr\nsbbu33Pyza25uZlFixZx7733MmHCBB577DHa2tpQq9X84x//wMPDo+OxBw8e5MEHHyQ4OBiAkJAQ\nnnzySXNFF0KIfqNFb+Bwlo59x4rJOF2FCdColYyP8GJS5CDCA1xQKvtn846+MiAKtBA/J+xtNBzO\n0nHL7JB+25FFCCHMKTAwiIoKHaWlJdTV1bFnTwLu7p48+eRfycw8zuuvv2zuiP3af//7X5ycnAB4\n+eWXWbZsGQsWLGDNmjWsXr2aRx55pNPjx44dy6uvvmqOqEII0a+YTCayC2rYd6yYxMwymlsNAAQP\ndmJS1CBiQj2xtR4QZUmPGBDvhEqpJDrYnb2pxZwsqmWYr5O5IwkhxAVdP2zRBUe7zqUn5tdPmDCZ\nN9/8D1OmTKO6uoqgoPYRnu++20VbW9sVHXsgy83NJScnh+nTpwPw9NNPY2VlBYCLiwvp6elmTCeE\nEP1TeU0T36eV8P2xEsqq26cwujlaMTvGj4lR3v1yDbL+YEAUaNC+aPXe1GIOn9BJgSaEEOcxbdoM\n7r57Fe+9t5bm5ib+9ren2bVrB0uXLmPHju1s3fqFuSP2S88//zxPPvkk8fHxAB1LEBgMBj7++GN+\n85vfdHlOTk4Od999NzU1Ndx3331MmjSpTzMLIcTlMhpNtLYZaG0zotcbaW0zoG8z0nr2dmub8ez9\ns9vbjOjbDLTqz25vM1BU3kBmXjUAWo2SiZHeTIr0JjTARWa7XcSAKdCGD3HBWqsiOauMG2cEoZC/\neCGE6CI8PILvvjvYcX/NmvUdtydPngbAwoXX9nmu/iw+Pp7o6Gj8/Pw6bTcYDDzyyCOMHz+eCRMm\ndNo3ZMgQ7rvvPubPn09+fj63334727dvR6vVXvBcLi62qNWqK87s4eFwxccwB0vMbYmZwTJzW2Jm\nsJzcDU16Xll3hMTjJbQZureu2MVEBLoxK9aPiSN8+qTFvaW81xczYAo0jVrFiCA3DmWUkV9Wj7/X\nwPgLEkIIYV4JCQnk5+eTkJBASUkJWq0Wb29v4uPjCQgI4L777uvyHC8vLxYsWACAv78/7u7ulJaW\ndinyfq6qqvGK81pqq2lLzG2JmcEyc1tiZrCc3GVVjbyyPpXiikYGe9pjb61Gq1GhUSvRqpVo1Kr2\nrxolVmoVGo0Srfpn+zWdbzvYaHCyb58K3lDXTENdc6++Bkt5r39woWJywBRoAGNCPTmUUcbhLJ0U\naEIIIXrEyy//2Dzltddew9fXl/LycjQaDQ888MA5n/PFF1+g0+m488470el0VFRU4OXl1VeRhRCi\n207kVfH6xmM0NLcxJ9aPe5eNorKi3tyxrmoDqkCLCnRFrVKSnKVj8ZRAc8cRQggxQH388ce0tLRw\n2223ARAUFMSf/vQnHnroIZ599lni4uL4/e9/z86dO9Hr9fzpT3+66PRGIYToa7uPFvHh1ycA+MX8\nMKaO9EF1lbe47w8GVIFmrVUTOdSVlJxySiob8XaVzjBCCCF6zv333w/A9ddff879L730UsftN954\no08yCSHEpTIaTXy6K4ftifnYWau57/ooQv1dzB1LnKU0d4BLVVRfwh+2/Y3C+uJz7h8T2r5Q6OEs\nXV/GEkIIIYQQot9ramnj1Q2pbE/MZ5CbLU+ujJHirJ+xuAKtsa2JMzWFJOTvO+f+kcPcUSoUJJ+Q\nAk0IIYQQQogf6KqbeObDZFJzK4gc6soTt8XgKWuR9TsWV6AFOgXgYuPEEd0x9Maui6ra22gIC3Dm\nVHEtlbW92y1GCCGEEEKIy2Uy9Uw7++7Iyq/mr+8nUVjewKwxg3nwxhHYWg+oq50GDIsr0JQKJZP8\nYmhqayKj4sQ5HzMmRKY5CiGEEEKI/utIto4HXtnDMx8ms+9YMa16Q6+da9+xYv75yRGaWtq4fW4o\nN88OQaW0uDLgqmGRfzOTAmIBSCpNOef+USEeKJACTQghhBBC9C8mk4ltB/N4fcMxWvQGcgtreGdr\nBg+/vo+Pv8miUNdzLe6NJhOfJeTwztYMrDQqHl42kumjfHvs+KJ3WOS4ZqCLP5427qSWH6e5rQVr\ntVWn/c72VgT5OnEiv5raxlYcbaW1sRBCCCGEMK82g5GPtp9g99FinO21PHjDSGyt1ew+WsTe1GJ2\nJBewI7mAYYOdmB7tQ0yoJ1qN6rLO1dzaxlubj3MkuxwvV1sevGGEdDi3EBZZoCkUCsZ4RfPV6R0c\nKz9OrPeoLo8ZHeJBTmENKdnlTB3pY4aUQgghhBBCtKtv0vOfTcfIzKsmwMuBB24YgYtD+yDD0mlB\nXDd5KEdzyklIKSL9VCU5BTV8/E02EyO9mTbKF193u26fq6KmmVc3pJJfVk94gAv3LonEzlrTWy9N\n9DCLLNAAYs4WaEmlR85doIV68OmuHA5n6aRAE0IIIYQQZlNa2cjL61MprWxkdIgHv1o0HCtt55Ex\ntUrJmFBPxoR6UlbdxJ6jRez5yaha8GAnpnVjVC2nsIbXN6RS26hnxihfbpoVjFplkVc1XbUstkDz\ntvPEz96H45VZ1OsbsNd0/lTB09kGf097jp+upLG5TbrUCCGEEEKIPpd5pop/bzpGQ3Mb88f7s3Ra\nEEqF4oLP8XS26RhVS8ku57uj7aNq2QU1rN2RzYRIb6ZFdx1V259ewuovMzEaTdwyO4SZYwb35ksT\nvcSiq5YxXtHk535JStkxJvuO77J/dKgHeXvqST1Zzvjh3mZIKIQQQgghrlbfHDzDv9cfBeCOBWFM\nGXFps7rUKiUxYZ7EhP1sVC2pgB1JP46qjQn1ZOv+02z5/gw2VmruWRxB5FC3XnhFoi9YdIEW4xVN\nfO6XJJWmnLNAGxPiQfyeUxw+oZMCTQghhBBC9AmjycT6hFy2HczDzlrNfddHEervckXH7DKqllJI\n+ukqsgtqeO+rE7QZjHg62/DgjSMY5Nb969VE/2PRBZqLtTNBTkPJqT5FVXM1LtbOnfb7uNvh5WpL\n6skKWvWGy+6CI4QQQgghRHe0tBp4c3M6R7LL8fWw474lUXj1YPfETqNqVY3sPlrMvmPF+HrYcfd1\nkdjbSDMQS2fxVwzGeEVjwkRy2dEu+xQKBWNCPGjVG0k/VWmGdEIIIYQQ4mpRWdvMs2uSOZJdTniA\nC/98YGqPFmc/5+liyw3Tg3jp/sn8bnm0FGcDRLcKtBdeeIHly5ezdOlStm/f3mlfcXExN910Ezfc\ncANPPfVUr4S8kNGeI1AqlCSfZ9HqMaEeACTLotVCCCGEEKKXnC6p5W8fJJFXWs/UkT48tGwk9n24\nFq/iIo1HhOW4aIF24MABsrOzWbduHW+//TbPPPNMp/3PPfccq1atYv369ahUKoqKinot7LnYa+0I\ncw0mr66Q0sauRdgQbwdcHKxIyS6nzWDs02xCCCGEEGLgSz5RxnMfHaamvpVlM4axcl6otLYXl+2i\n3zmxsbG88sorADg6OtLU1ITBYADAaDSSnJxMXFwcAE8//TQ+Pn2/5lisV/s6aEnnGEX7YZpjY0sb\nJ/Kq+zqaEEIIIYQYoEwmE1v3n+bfm9JQKBTctzSKeeP8ZTRLXJGLNglRqVTY2rbPnV2/fj1Tp05F\npWpvtlFZWYmdnR3PPvss6enpxMTE8Lvf/e6Cx3NxsUWtvvJmHR4eDh2345zH8fGJDaSUp7IydkmX\nfxRx4wLYkVzA8bxqpo8NuOJzX4mf5rYklpjbEjODZea2xMxgmbktMbMQQgxEbQYj72/LZN+xElwc\nrHjwhhH4e13a/9EGowGVUprYic663cVxx44drF+/nnfffbdjm8lkorS0lNtvvx1fX1/uuusuEhIS\nmD59+nmPU1XVeEWBof0XFJ2urtO2SLdwjpSlcvhUJv4OnRfl87TX4mCrYV9qEUunDEWpNM+nGufK\nbQksMbclZgbLzG2JmcEyc1tqZiGEGGjqGlv596Y0svKrGeLtwAM3jMDZ3uqSjnG6No/XU95h+uCJ\nLAqc20tJhSXqVoG2Z88e3njjDd5++20cHH78Yevi4oKPjw/+/v4ATJgwgezs7AsWaL0l1iuaI2Wp\nJJWmdCnQlEoFo4Ld2X20mNyiGoIHO5/nKEIIIYQQwtIZjEYqalsoq2qktLKJsqomSqsaqWloRUH7\nJTBK5dmvHfcVKBTttxUKUCoUKBVdtykUcLKolvKaZmJCPbhz0XCsLnEpp6a2ZlanfUxTWxPbTn9L\nmGsIw5yH9sp7ISzPRQu0uro6XnjhBd577z2cnTsXNmq1Gj8/P06fPs2QIUNIT09n4cKFvRb2Qoa7\nhWGjtia59CiLgxagVHS+vG50iCe7jxaTfEInBZoQQgghhIUzGI1U1DSfLb7aC7AfbpdXN2Ewmro8\nR6tWggKMxvaZYCbT2a+XeG4FsHBCAEumBqK8jOvN1p2Ip7y5kpEekaTq0vkw41MeH/sQVqq+6/oo\n+q+LFmhffvklVVVV/Pa3v+3YNm7cOEJDQ5k9ezaPP/44f/zjHzGZTISEhHQ0DOlrGqWaaI8o9hcn\nklt9mmCXwE77wwNcsLFScThLx/K4YXLxphBCCCGEBaiqayGvopGsUxUdBVhZVSPlNc3nLMLsbTQM\n8XbA08UGLxfb9q+utng425x3nbAfijVjp68/FnDGn+wzmUxo1ErsrC9vzbFDJYdJLD1MgKMfd0bc\nwuaTX/NNXgLxOV+yPHTxZR1TDCwXLdCWL1/O8uXLz7s/ICCAtWvX9mioyxXjFc3+4kSSylK6FGga\ntZKRQe4cOF5KXmk9Ad5yXYQQQgghRH9iNJooLG8gp6Ca7MIacgpqKK9p7vI4B1sNQwY54Olsi5er\nTadi7HIKp44pjPTuB/jlTRWsO7EJK5WWO4bfjEqpYuHQ2aRVZLC78HtGekQQ5hrcqxlE/9ftJiGW\nIMQlCAetPUfKUlkWfF2XrjijQzw4cLyU5CydFGhCCCGEEGbW0mrgZHEt2QXV5BTUkFtUQ1OLoWO/\nvY2G6GHuRAxzx16rai/GnG2wvczRK3MyGA28m/4xzYYWbg9fjoetGwAalYbbw5fzj+TX+SjjM54Y\n9zA2amszp716mUwmWgwt1Osb2v+0NtCgb+y436BvYKRHJBFuYb2WYUAVaEqFkjGeI0ko2EdGZRaR\n7uGd9kcFuqFVK0k+Ucb1UwPPcxQhhBCiq+bmZhYtWsS9997LhAkTeOSRRzAYDHh4ePCPf/wDrbbz\ntSPPPPMMR48eRaFQ8PjjjzNixAgzJRei/6iqayGnsKajIMsrrcdo+nGaoperLWNCnBg22IngwU54\nu9qiUCgssovtz2099Q1navOtknneAAAgAElEQVSJ9RrFuEFjOu3zdxzM3IA4vjq9g43Zm7kl/EYz\npRx4Wg16GvQ/FlkN+gbq9A00tDZQr2+k4YdCrKMYa6DNZLjgMfXGNinQLkWMVzQJBftIKj3apUCz\n0qqIDHTjcJaOovIGfNztzJRSCCGEpfnvf/+Lk5MTAK+++io333wz8+fP51//+hfr16/n5ptv7njs\noUOHOHPmDOvWrSM3N5fHH3+cdevWmSu6EGZhNJko0jWcnapYTfbPpiuqlAqG+jgQ7OvMsMFODPN1\nwtFuYDbJyKrKYfuZXbhZu7I8dMk5HzNvSBzHyo/zfXEiIz0iu/wee7VrH9lq7Si2fiy42r8a8/SU\n11af3fdj8dVq1Hfr+NYqa+w1tvg6+GCvsev0x05re/a2PXYaWzxt3Xv1tQ64Am2Ioz9u1q4cLU+j\n1dCK9mfdcMaEeHA4S8fhLJ0UaEIIIbolNzeXnJycjmVkDh48yJ///GcAZsyYwbvvvtupQNu/fz+z\nZs0CICgoiJqaGurr67G3t+/z7EKYQ1Z+NW9vOd6pILOzVjMyyO3s6JgzQ7wd0F5ie3pLVK9v4P3j\n61AoFNwRcdN5py+qlWpuH76c5xNf5ePM9Twx7nfYaWz7OG3/UlhfzNrMjVQ2V3VrZOsHWpUWO7Ut\nXnae2KltsdfaYaexxe5nhddPt2uU/acs6j9JeohCoSDGK5qvz3zLsfIMxniN7LR/5DA3VEoFyVk6\nFk0cYp6QQgghLMrzzz/Pk08+SXx8PABNTU0dUxrd3NzQ6XSdHl9eXk5ERETHfVdXV3Q63UULNBcX\nW9TqK/+F1VIXCLfE3JaYGXovt8FgZN2OLNZ9cwKA6aMHExnkRvgQVwZ7OqBUXn4TDkt8r00mE+tP\nxlPdUsOKqGsZOyzygo/38HBgWdMi1h77nC/yvuSB8Xf0UdKuOcytpa2Vvyd+TFFdKV527njYu+Jg\nZYeD1h57KzsctHY4WNl3bPvpPq3K8q5R/KkBV6ABHQVaUmlKlwLN1lpDeIALaacqKa9pwt3Jxkwp\nhRBCWIL4+Hiio6Px8/M7536T6eIrKHXnMQBVVY2XlO1cLPVaHUvMbYmZofdyl9c08ebm4+QU1ODm\naMVd10Z0Wnu2oqL+so9tqe/1kZojJBYeJdg5kEnuE7v1Gia4jed7x8PsPXOIcMcwoj0uXNT1tP7y\nXq87sYmiulLi/KawNPiaiz7ew7k9d01DM9C182d/c6EieEAWaD723vjYeXO8IpNGfRO2ms5F2OhQ\nD9JOVXI4q5w5sef+gSuEEEIAJCQkkJ+fT0JCAiUlJWi1WmxtbWlubsba2prS0lI8PT07PcfT05Py\n8vKO+2VlZXh4ePR1dCH6zKGMUt7fdoKmljZiwzxZOS/UIjst9qSi+hLeT1mPndqWlcNXoFQou/U8\nlVLF7eHLeS7xZdZmbiDIaQgO2qtrenRaeQa7C/fjY+fNtYHzzB2nz3XvO8UCxXhF02YykKJL67Jv\nVLAHCuDwibK+DyaEEMKivPzyy2zYsIFPP/2UG2+8kXvvvZeJEyfy9ddfA7B9+3amTJnS6TmTJk3q\n2J+eno6np6dcfyYGpObWNt7Zepw3Pk/HaDRxx4Iw7r4u4qovzvQGPavTP0Zv0HNL+A24WDtf/Ek/\n4W3nyTWB86jXN/DJiU3dHoUfCOpa6/ko8zPUChW/iLgJjYVPV7wcA7ZAG+MVDUBS6ZEu+5zstAQP\ndiK7oIaahta+jiaEEMLC3X///cTHx3PzzTdTXV3N4sWLAXjooYdobm5m9OjRREREsGLFCv72t7/x\n9NNPmzmxED3vVHEtf16dyL5jJQR4O/D0HbFMGeGDQtG7iz1bgk25X1LUUMLsoCmMvMwpijP8JhPk\nNIQU3TGSS1N6OGH/ZDKZ+DhzA3Wt9VwTNA9f+0HmjmQWA3KKI4C7jStDHQPIqsqlpqUWJyvHTvtH\nh3qSVVDDkSwd00f5mimlEEIIS3L//fd33F69enWX/S+99FLH7d///vd9kkmIvmY0mfj6UB4bvzuJ\nwWhi3jh/rp8aiFo1YD/3vyTHyo/zXcE+vO28uD36BmqrWi7rOEqFktvCl/PMoX+xLiueYJegLr/P\nDjTfFx8itTydEOcg4vymXPwJA9SA/pcU4x2NCROHy1K77Bsd0r5+QZJMcxRCCCGE6Jbq+hb+tS6F\nz3blYm+j4XfLo1k2Y5gUZ2fVtNTyUcZnqJVqVkXcjJX6ytZ187B1Y8mwhTS2NfFx5oYBPdWxrLGc\n9dmbsVHbcPvw5d2+Zm8gGtCvfLTnCBQoSDrHsLC7kw1BPo5knKmiVqY5CiGEEB3ST1Xy3leZNLe2\nmTuK6EdSsst56p1DHD9dxcggN/5851gihrqaO1a/YTQZ+eD4Our1DSwJWthj0/Mm+44n1GUYaRUZ\nHChO6pFj9jcGo4H3j39Cq6GVFaFLLvmavYFmQBdojloHwlyDOV2bh66xosv+2HAvTCZIllE0IYQQ\nokPCkUJ2Hy3irc3HMQ7gT+xF97TqDazZnsWrG1JpbjVwy+wQHrhhBI62VzY6NNDszNtNZlU2kW5h\nTBs8sceOq1QouTX8RqxVVqzP3kxVc3WPHbu/2HbmW07X5hHrNYqYs30krmYD9hq0H4zxiiajMovk\nshTmDZnZaV9smCfrdmZzKKOMGaMHmymhEEII0b9UN7RfM3Mku5xNu0+ydFqQmRMNXGknKzhwvLRH\npq4pFQrcnW3wdrXF29UWL1cbrLVX9qtega6e/32RTqGuAR93O359bQR+ntKR9OfO1ObzxcltOGod\nuDV8WY83SnG1dmFp8DWsyVzPRxmfcV/0LwdMM5ZTNWfYdnonLlbOLAtZbO44/cKAL9CiPSL45ISa\nxNIU5gbEdfpmdnGwIniwE1n51VTVteDiYGXGpEIIIUT/UFPfir2NBltrNVv3n8HH3Y4JEd7mjjWg\nGIxGNu4+yVcH8nr1PC4OVj8p2Nq/RiiVKIwmlMrz/4JvMpnYdaSQdd/moG8zMmOUL8vjhqHVqHo1\nryVqbmvhvfS1GE1Gbh++vNfWLJswKJYUXRrpFZnsLTrAFN8JPXr8VkMrh8tSsa+zIsI+sk8KwOa2\nFt47/gkmk4mVw5d3Wbv4ajXgCzQbtQ2RbmGk6NIoaijpMh84NtyLrIIakjLLmC2LVgshhLjKmUwm\nahpa8XW341fXDOdvHySz+stMPJ1tCPJ1Mne8AaGqroU3Pk8ju6AGTxcb7lwYjpuj9RUfV28wUlbV\nREllY/ufikZKqxrJOFNFxpmqTo9VqxR4uth2Gm0b5GqHt5stJpOJ1V9mkpJTjp21mruvjWBUiCy0\nfj6fZX1OWVM5M/2nEu4a0mvnUSgU3By2lL8f/Bcbc7YS7hqCu43bFR+3qL6EvUUHOVSSTFNbM9De\nx+HW8GVYqXp3GuvGnM2UN1Uw2386wS4yUv+DAV+gQfs0xxRdGkmlKV0KtJgwTz7ekcWhzFIp0IQQ\nQlz1mlra0LcZcba3YpCbHfcsjuClT4/y2sZjPHl7DG5OV15IXM3STlXw5hfHqW/SExPmyR3zw7Cx\n6rlfx7xcbIkK7PxLe0urgdKqxo7CrbpBz+miGkoqGykqb+hyDKVCgdFkIjzAhV8uGi4zjC4gqTSF\nAyVJ+Dv4cm3gvF4/n7OVEzeGXMf7xz/hw4xPeXDUry+r26HeoOeI7hh7Cw+QW3MaACetA9MCJnK6\n4QyHy1IpbdRxV9RK3G16pxHMUV06+4oOMdjeh4WBc3rlHJbqqijQIt3CsVZZkVSawrWB8zoN2TrZ\naQnzdyHjTBUVNc3yg0cIIcRVrbq+vbOxo137J+eRQ91YMTOYtTuyeW1DKo/dOgYrrUxzu1RGo4nP\n955iy/enUSoV3DI7hLjRvn0yjcxKq8LfywF/LwcAPDwc0OnqMJlM1Da0/jjidnbUraquhXHDvZg7\n1v+C0yCvdhVNlazN3IhWpeUXETejVvbNr9WxXqNI0aVxVJdGQsG+S1ovrLShjL1FBzlYnExDWyMA\n4a4hTPYdT5RbOCqlChdXG97Y/zG7C/fzQuKrrIq8hTDX4B59DTUtdXycuR6NUs0vIm5C00fvnaW4\nKt4NrUrDSI9IDpYkc6r2DIFOQzrtjw33JONMFYmZZcwb52+ekEIIIUQ/UFPf3iDE2f7HqU2zxgym\nqLyB71KKeGvLce5dEolygDQo6As19S3874t0MvOqcXey5p7FkQwdZP4FhxUKBU72VjjZWxHq72Lu\nOBbFYDTw3vG1NBuauTXsRrxs+24KqEKh4KbQ68mtPsUXuV8R4RqKl53neR/fZmzjqC6dvYUHyKrO\nBcBeY8ds/+lM8hmHh23nEVe1Ss3y0CUMdvBh3Yl4/n30HZYMW8iMwZN75AMFk8nER5mfUq9v4Mbg\n6xhk53XFxxxoBnSb/Z8ac7Zl57nWRBsT4oFSoeBQRmlfxxJCCCH6lZqza4M62f84rU2haB/xCfN3\n5nCWjvg9J80Vz+JknKni6dWJZOZVMyrYnafviO0XxZm4Ml+d3snJmjOM8RzJ+EExfX5+B609y0OX\noDe28WHGpxiMhi6PKW+q4PPcr/i/fc/wbvoasqpzCXEOYlXEzfx90hMsHragS3H2U5N8xvHb0Xdj\nr7FjQ/ZmPshYR6tBf8XZ9xQe4HjFCcJdQ5g6uGcbnQwUV8UIGkCYyzDsNXYcLk1l6bBrUCl/nJ7h\nYKtl+FAX0k5WUlbViKeLrRmTCiGEEJcnp/oUxQYNg1SXv3TMD1Mcne06NwdQq5TcuySKv72fxJbv\nzzDIrW86Oza1NbOncD8HSw4zx3864waN6fVz9gSjycTW708Tv/cUSoWC5XHDmBPrN2Bao1/NUnXp\nbDu9E1drF1aEXm+2v9PRniM46hVNUmkKO/N2M2fIDAxGA8cqMthbeICMyiwA7NS2xPlNYbLPuAuO\ntJ1LoFMAj8Y+wFvHPuRQyWFKGkq5K2rlZS8kXdJQxsacLdipbbk1/MbLun7uanDVFGgqpYrRniPY\nXbifrKpcwt06d9kZG+ZF2slKEjPLWDhhiHlCCiGEEFdg88ltnKkr4JmJT2CrubwPG2vOroHmaN+1\ne5u9jYYHbhjB3z9Mau/s6GJDkE/vdHasa61nV/5edhd+39FZbk3mejxs3Ql0CuiVc/aU2sZW3tp8\nnPRTlbg4WHHP4kiGSQfMAWFf0UHWZm5ErVRzR8RNZm8LvyxkMVlVuWw9tZ06fT3JpSnUtNYBEOQ0\nhMm+4xnlEYVGpbnsczhbOfHb0Xez7sQm9hcn8lziK/wy8jaCXQIv6ThtxjbeO74WvVHPL4avwNlK\n/k2cz1VVtsZ4jQIgsfRIl32jQ9xRqxQcPF7W17GEEEKIHhHhFobeoD/ndP7uqukYQTt35z4fdzvu\nvi4Sg9HIaxuOUVnbfNnnOpfK5io+zfqcJ79/lq/PfItKoeLawHn8KvI2jCYjbx/7kJqWuh49Z0/K\nyq/mz6sTST9VSVSgG3+6I1aKsyugN+jZX5zEtye/75HFvC+XyWTiy1Pf8HHmBmw1Njw46q4uPQ3M\nwU5jy81hS2kzGfg2fw+tRj3TBk/iibEP8/CYexnrPfqKirMfaJRqbgm7gWUhi2lsa+LVlDfZXXBp\nfydfntpBfl0h4wfFEO0ZdcWZBrKrZgQNYKiTPy5WzhzVpaM36Dt9w9paa4gc6kZKTjnFFQ0McrMz\nY1IhhBDi0o3zHsPmk1+zvziRqYMnXtYxfrgGzdHu/OsfRQW6sSIumLU7s3l1fc90diyoLWbd8a0k\nlh7BaDLiYuXMrIBpTBwUi/bsWkzXBc0nPvdL3kn7kAdG3dVnXfO6w2gy8fXBPDZ8dxITJpZOC2T+\n+ABppnKZ6vUN7Ck4wHcF+6jT1wMwwj2C28KX9fmolcFoYF3WJvYVHcLN2oXfjLzzkqcK9qYo9+Hc\nHr4cIybGeI7o+PfS0xQKBdMGT8THzou30z5iXVY8+XWFLAtdctEujDnVp9h+Zhdu1q7cGHxtr+Qb\nSPrP/2x9QKlQEuMVzTd5CaRXZHap3mPDPUnJKedQRhnXTR5qppRCCCHE5XGycmTUoAiSi45RUFfE\nYAefSz5GdX0LdtZqNOoLT7KZFTOYwvIGdh8t4u0tx7nnMjs7nqnNZ/uZXRzVpWPChLetJ3MCZhDj\nFd3penGAWf7TyKsr4HBZKhtztrAsZPEln6831DfpeXvLcVJzK3Cy13L3tRHSFfEylTWWsyt/D/uL\nk9Ab9VirrJnlP42S5hJSy9J5PvEVfhl1G34Ovn2Sp9XQyrvpazhWnsFgex/uHbkKJ6v+1+SlL6/N\nDHYJ4tHYB3jz2Ad8X5xIcUMpv4y67bxTFpvamvng+CcArBy+Amu1LGl1MVdVgQZ0FGiJpSldCrTo\nYe5o1EoOZZRy7aQhciGvEEIIixMXOInkomPsL07kRofrLvn5NfWt3VqYWKFQcOucEMqqGkk+29nx\n+qlB3TqHyWQiqyqX7Wd2kVmVDUCQawBxvtMY4T78nI0DTCZTezfJsBspaSjju4Lv8XcYbJYOej+V\nW1TDG/FpVNS2MHyIC3ddE3HB0UdxbidrTrMzb3dHoe5i5Uyc32Qm+IzFRm2Nm5sd7x3awLYz3/LP\n5H+zPGQxE33G9mqm+tYG3khdzanaPMJcgvll1G3YSHEBgKu1Cw+PvpePMzeQWHqY5xNf5VdRt5/z\n+tDPsj6normKeUNmEuQ8pO/DWqCrrkDztR+Et60naRUZNLU1d/qHZmOlZkSgG8lZOgp1DQz2tDdj\nUiGEEOLSjRoUiYPWnsSSIywetvCSFoBt1RtobGljyCCHbj3+550dfdzsGH+Bzo5Gk5Fj5RlsP7OL\n07V5AIS4DGNuwAwmh4yivLy+0+NNJhPZBTXsOVpE0gkdGrUSV0crHJzHoXL8ijUZG6jSaQl2DcDV\n0Qpne6s+W1jZZDLx+e5cVm9Ox2g0sXjyUBZNHCILO18Co8lIqi6dHXm7OVV7BgB/B19m+k9jlEdU\npxFUpVLJNUHzGOoUwPvHP2FN5npya06zPGRxr0zpq2iq5N9H36G0UUes1yhuDb+xX02p7Q+0Kg0r\nhy/H38GHjTlbefnwGywPXcwkn3EdjzlclsrBkmT8HQazYMgsM6a1LFfdd5pCoSDGaxRbTn3NUV1a\nl0/eYsM9Sc7ScSizVAo0IYQQFketVDHOeww78r4jVZfOGK+R3X5u7Q9roJ2nQci5/LSz47tfZuJx\njs6OBqOBpNIUvslLoLihfc3Rke4RzBkygyGO/gCdZq3UNrTyfVoJu48WUVLZCICbozUatZKSikby\nSo0onSLRhhzmi8L1tGyfCG1alAoFLg5aXB2tz/6xws3RGleHs7edrLG1UqNQKDCaTLS0GmhuNdDc\n2kZzq4GmljaaWjrfb2410NTaRnPLTx7TaqChSU95TTOOthruujaC4UNcu/2eXe1aDK0cKE7i2/w9\nlDdVABDpFs4s/6kMcw684AymSPdw/hj7IG+nfcSB4iTy6wr5ZeStePbgQtH5dUX85+g71LbWMct/\nGtcFzZd28OehUCiI85+Kj/0g3k1bw8eZGyioK2Jp8DXU6xtYm7kBjVLDL4av6DJlWZzfVVegAYzx\nGsmWU1+TVJrSpUAbGeSOVqPkUEYZS6Zc+D8JIYQQoj+aMCiWHXnfsb848ZIKtOqORaovbUTih86O\nL392lNc3HOPJlTG4OlpjNBnZW3iQHXkJVDRXoVQoGes9mtn+0/Gx7zzSZjCaSDtZwXdHi0jJLsdg\nNKFWKRk/3IspI30I9XdGqVBgMpmob9JTWdvCzgIbkmv3MWh0JoPr4qiq1VNR20xOYQ2mgppzZrXS\nqFAqobnFwOX2BNRqlFhr1Ywd7s2KuCCc7btf0F7Nalrq2F2wjz2FB2hoa0StVDPJZyxxflPxvoSm\nG242rjw85l42ZG9mT+F+nk98ldvCl/VIZ8DMymzeOvYBLYZWbgi+lhl+k6/4mFeDMNdgHom9n/+l\nvs/uwv0U1hejUqhobGtiReiSftVUxRJclQWap607AY5+nKjKoa61HgftjyNlVloV0cPcOZRRRl5p\nPQHe3ZvmIYQQQvQX3naeBDoFkFmZTWVzFa7W3WtYUVPfvgbazxep7o6oQDeWxwXzyc5sXt2QymO3\njCGh6Du+OLkNjVLNVN8JzPKfhptN55Gmippm9h4r5vv0EnRVTQD4etgxdaQPEyK8sbfp3CJcoVDg\nYKvFwVbLL7yuofVYFcfKjzMmIo97hi0CwGA0Ul3XSmVdMxW1zVTWtlD5k68A1loV1lZqrLUqbKzU\n2GjVHdtsfvpVq8baStW+30qFtVaFStk+muLh4YBO139b/vcXxQ2l7MzbTWLJYdpMBuw0tswfMotp\ngyd2+h3sUmiUalaELiHQKYC1mRt4K+1D4vymsDhowWWP1CSWHOHDjE9RAHdE3HxJH24IcLdx4/cx\n9/FhxqccKUsFINItjMk+482czPJclQUatDcLOVObT3LpUab7Teq0b2y4F4cyyjiYUSoFmhBCCIs0\nYVAsJ2vOcKA4iQVDZ3frOdX1P4ygXd6I0OyYwRSd7ez45pZUCjz2YKO24f/GPdypw1ubwcjRnHK+\nO1pE+slKTICNlYqpI32YOtKHoYMcujWDRalQsnL4cl5Ieo2debvxdxh8tvujEjcna9ycrAm+rFci\nrpTJZCK7+iQ78r4jvSITAE8bd+L8pzDOe0yPXTc21ns0g+19eDvtQ77N38Pp2nzujLzlkhZBNplM\n7MzfzaacrdiorbkraiUhLt1reCM6s1JpuTPiFnY6+pFZmc0t4TfKbLTLcNUWaGM8o/k850t25e9h\niu/4Tp+2RAW6YmOlIjGjjBunB8k3lhBCCIsz2nMEn2V/wYHiJOYNmdmta2h+WAPN+RKnOP7gh86O\npZWNpFYfRevcwNyAuI5flosrGtiTWsz3x4qpbdQDEOTjyJSRPiyYEkR9bdMln9NGbcOvo1byQtJr\nrMn4jEF2XvjaD7qs/KJnZFflsuXUdnKqTwEQ5DSEmf7TiHIP75VruXzsvXkk5n4+ztxActlRnj30\nMndE3EyY68XLc6PJyMacLezK34uT1pHfRN8p3z9XSKFQMMt/GrP8p5k7isXqVoH2wgsvkJycTFtb\nG7/+9a+ZM2dOl8e8+OKLpKSk8OGHH/Z4yN7gZOXABJ+x7CncT1JpSqf1IzRqFdHDPNifXsLJolqC\nfLv/KYwQQoiBp6mpiT/+8Y9UVFTQ0tLCvffey6ZNm6iqqgKgurqa6Oho/vrXv3Y8Z+PGjbzyyiv4\n+7c3wZg4cSL33HNPn2W2Vlsz2nMEB4qTyKrK7dYvqz9McbySNvFqlZK7lwznid1fYDQqsakL4vu0\nYnanFJF19rowO2s1s2P8mDJyEIM92qe42Vipqb/QgS/A286L28OX81bah7yZ+j6Pxj6Arcb2sl9D\nX9Eb9BwuS6WsUUe0Z1Sfre3VW3KqT7H15HayqnMBiHALY/6QmQw9R+v1nmattuaOiJsJdB7Cxuwt\nvJ7yNosC5zAnYMZ5i0K9sY0Pj68juewo3nZe/Gbkqm5PBxaiN120QDtw4ADZ2dmsW7eOqqoqlixZ\n0qVAy8nJITExEY1Gc56j9E9zAqbzfdEhtp3eSaz3qE7/gMeGe7I/vYRDGWVSoAkhxFVu165dREZG\n8qtf/YrCwkJWrVrF119/3bH/scce48Ybb+zyvAULFvDoo4/2ZdROJg4ay4HiJPYXJ3avQOsYQbuy\nphfZtRmYtI0oyv1Zm5TfsX34EBemjvRhVLDHRRfCvlTRnlHMDYjj6zPfsvr4Wu4ZcUe/7bxX1VzN\n3sID7C06SL2+AYBtZ77F38GXiT7jiPGKtqj1tk7WnGbryW861rQb7hrKgqGzGerk36c5FAoF0wdP\nIsDBj3fSPmLzya85WXOGlcNXYPezgr2prYn/pb5PdvVJgpyG8OsRv+jyGCHM5aIFWmxsLCNGjADA\n0dGRpqYmDAYDKtWPUwKfe+45HnroIV5//fXeS9oLXK1dGD9oDPuKDpFcepRY71Ed+yKGumJnrSbp\nRBnLZw5DKdMchRDiqrVgwYKO28XFxXh5eXXcP3nyJHV1dR0/K/uTQKcAvGw9SNGl0ahvvOioUnV9\ny9kOhZffDttkMrE9LwEFCm6Ons83jZVEBboyeYQPns42l33c7lgUOIe8ugKOV5xg66lvuCZwbq+e\n71KYTCZya06TULCPo7o0jCYjtmobZvlPI8DRj0MlyaSVZ/LJiY1szNnCGM+RTPIZxxBHv357qcWp\nmjNsPfUNGZVZAIS5BLMwcM45FyvuS0Od/Plj7IO8d3wt6RWZPJf4Cr+MvJUARz8Aqltq+HfKOxQ1\nlBDtEcnK4TehVVnWIIMY2C5aoKlUKmxt2/9DX79+PVOnTu1UnG3cuJGxY8fi62uZw/JzAuLYX5zE\nttM7GeM1suPTNrVKyagQD/amFpNTUEOIn7OZkwohhDC3FStWUFJSwhtvvNGx7YMPPuDWW2895+MP\nHTrEnXfeSVtbG48++ijDhw+/4PFdXGxRq698rSAPjx8bXM0aNpk1qZvIbMhkbvCFrwmpb9Lj5miD\np6fjZZ87pfg4hfXFTPQbw3UTo7lu0sWf84Of5r5cf5h6F4998xzbTu8k0ncYYwdHX/ExL+RimVvb\nWtmbl8hX2QmcqS4AIMDJl3nB05kcMBYrdft00rkRk6hsrGbXqe/59uQ+9hcnsr84EX8nX2YGTmLK\nkLHYa+36LPeF5FSc5rP0LRwpTgcg0jOUZZGLCPMY1lPxzulSMnvgwNM+D7Lx+DY+S9vCvw7/l5XR\nNxDhGcJLB/5LeWMlc4ZNZdWo5SiVvTvS2hPf133NEjOD5eb+OYXJZOrWMiA7duzgf//7H++++y4O\nDu0vvrq6mvvuu4/Vq1dTWlrKY489dtFr0NraDD3yw6cn/efQBySc2s9vJ/ySif4/Xot2+EQZT7+5\nn4WThnL39f3vk1EhhIKsYyoAACAASURBVBB9LyMjg0ceeYQvvvgCvV7P0qVL2bx5c5fH5ebmkp+f\nz/Tp0zly5AhPPfXUOR/3Uz3Rsv3nrd9rWur4v+//jq/9IP4Y++D/s3fnYXGWZ9/Hv/csMAzbzMAM\nS9h3AiEhqzFmNUlj1NbWatJUbau11uWp2tYtHkbbvtUaa/vUto+2rq1tNW1qrdVoFo2amI0QkgBZ\nIISwhWXYhn0b5v0DGKVZIAkwM3B+joNjhnuZ+UHIMCfXdZ/XOc/r7XVw+zPbiZ8UyNqbZpzzuKH8\n5sAfKGgs4uFZ917QNVUj2bK+vPk0v8z+PWpFxQMz/+eC1ti6EOfLXN/RwKflu9l1eh+tPW2oFBUZ\nwWksirh8yMWYex29HK8/wWen93KoNp9eRy9alYZp5gzmhc8mwRB7SaNqF/u9Lm0q573ireTVHQUg\n0RDH1bHLSTTGXXSW4bqUn4+j9QW8lv8GLd2taBQ1PQ47X45bwfLoxaM+OumJSzF4YmbwvNznKyaH\n1SRkx44dvPDCC7z00kvO4gz6rk+rr6/nm9/8Jl1dXZSWlvLkk0+ydu3acz5WQ0PbBUQ/u5H+B1gY\nMp9PT+3l74ffJV6X4BxFCzd44+ejZUdOOdfNi3aue3KxPO0HZ4An5vbEzOCZuT0xM3hmbk/NPB7k\n5eURFBREWFgYqamp2O126uvrOXbs2DmnNsbHxxMf39eqOzMzk/r6+jMuERgLgd7+pAWlkFt7hLLm\n00T6h5/1uOa2LhyOi1sDbcCpplIKGotIMSa6tOFFhH84N6V8nVePvMEfc//EAzP/Z0yu6eprLV/E\nx+W7OGzNx4EDP60vy6MXs2DSXIy64c3GUSkqUoOSSA1KormrhT2V+9l1eh9Z1QfIqj5AiN7M5eGz\nmRM646LXEbsQZc0VvFe8ldzaIwDEB8ZyTdwykoyjO2I2UlJNSTw8615ezvsrJc1l3Jx6I5eFzXR1\nLCHOacgCrbm5mfXr1/Paa69hMAx+YVmxYgUrVqwAoLy8nEceeeS8xZm7suiDmRWSyd6qbA5b850r\n0atVKmYmm/n44GmOlzYyOcY0xCMJIYQYj/bv309FRQWPPvootbW1tLW1YTQayc3NJSUl5aznvPji\ni4SFhXHNNddQUFCAyWQa8+JswNywWeTWHmF3ZRaR/l856zEDDUIudg00gK0lHwOwPHrxRT/GSJkZ\nmklJczkfle3g9SMb+O6Um0etaUinvYusqgN8Ur6L061VAET6T2JhxDxmWqaivYTrm/y9/FgWvYil\nUQs50XiSz07vI8eay79OvMc7RR+QYU5jXvhsko0JI/71VbRU8l7xVg5Z84C+axqvjl1OsjHBba+L\nOxejzsAPZ9xJe0+HNAMRbm/IAm3Tpk00NDRw3333ObfNmTOH5ORkli0b3sKXnuBL0YvZV3WATae2\nMdWc7nzhmZUawscHT7PvaI0UaEIIMUGtXr2aRx99lDVr1tDR0cG6detQqVRYrVZnG/0Bd955J88/\n/zzXXnstDzzwAG+++SY9PT38/Oc/d1F6SA9Kwd/Lj6yqA3w1fuVZC4aBRaovdg206tYaDlnzifKP\ncJtFfq+LX0l582kO1eazpWQ7K2KuHNHHr26x8nbhNnZVZtHe045KUTHDMpVFkfOIDYge0SJGURQS\njfEkGuO5ofsr7Ks6wK7T+8ipOUxOzWGCdCZmh04nwMsftaJC1f+hVlSoVOrP7ysqTHY/mps6UClf\n3K527m/vaefD0k/JseYCEBMQxdWxy0g1JXlcYfZFKkUlxZnwCEMWaKtWrWLVqlVDPlBERITHrIF2\nNiG+FmaETGV/9UFya4+QYU4DIDnSQKCvF9nHa7hpeRIatXu27BVCCDF6dDodzz777BnbH3vssTO2\nPf/88wCEhoa6ze9FtUrNnNAZbCv9hMO1+cwIObNxxqWugbat9FMcOFgWvcht3sSrVWpuTf8mT2c9\nx7sntxDpP4m0oLOPeJ6PvddOXUc9Va01VLdZqWqtoaqthpKmMhw48Nf6cVXMUq6YNMe5KPdo8tXq\nWRx5BYsi5nGqqbS/G/VB3j+1bUSfJ8o/gmviljPZlOw2/6ZCTATDugZtolgRcyXZ1Yd4/9Q2pgRP\nRlEUVCqFmSkWPswu52hJA1PiglwdUwghhLhgc8Nmsa30E3ZX7j9rgdZ4CWugNXba2FeVjcUnmGnm\n9EvOOpL8vfy4fcrN/OrA87ya/wYPzvwfLPrgsx7b0dNJTZuVqrYaqltrqOq/b22rxe6wDzpWpahI\nDIplbshsMi0ZaFVj/5ZKURRiA6OJDYzm+sRrKWwooru3G7ujl97+j8H37fQ6evHRa2lqaae3136W\nY+04gCnBqaQHpUphJoQLSIH2BWG+IWRapnCg5jD5dcdID04F+hat/jC7nH1Hq6VAE0II4ZFCfS3E\nBUZzrL6QuvYGgnyMg/Y39U9xDLyIEbTtZTvpcdi5MmqBWy4OHR0Qyerkr/GXo3/nxdw/c0fGt6nv\naOgfEatxjow1dDaeca5O7U2EfzihegshejOhvhZC9BbMPkGEhhjcpnGPj0bnnP0zFE9sOCTERCIF\n2n9ZEXMlB2oO8/6pD0kLSkFRFOInBWL09+ZAQS23fKkXrcb9fvkIIYQQQ5kbNpuTthL2Vu1nZezg\n68gbW/umOF7oCFpbdzs7K/YQ4OXPnNCLb88/2uaGzaS0qYxPK3bz+O5fnLHf4B1IsjHBWYANFGOB\nXgEyiiSEGFNSoP2XSX5hTDOnc9Cax7H6QlKDklApCrNSLGzJKiO/uJ5piWefGiGEEEK4s+mWKfyj\n8N/sqdzPipgrB4122Vq6UCkKfvoL6zi4s2IPHfZOvhSz5JK6FY6F6xOvpbu3h7ae9r4CTG8h1NeC\nRW8ekzb8QggxHFKgncWKmKUctOax6dRWUkyJKIrC7NQQtmSVse9otRRoQgghPJJOo2OGZSq7K7Mo\naCgixZTo3NfY0kmArxbVBYwWddu7+ah8Bzq1jvmTLhuNyCNKo9JwU+oNro4hhBDnJXP1ziLSP5wp\nwZM5aSvheMMJAGLD/AkO1JFzopaubvsQjyCEEEK4p7lhswDYXZnl3OZwOGhq7brgNdD2VmXT3NXC\n/EmX4aPxGdGcQggxUUmBdg5X9a+VMtCyVlEUZqVa6Oyyc7iozpXRhBBCiIsWFxhNiN7MQWsebd1t\nALR32unq6cVwAQ1Ceh29bCv9BI2iZnHkFaMVVwghJhwp0M4hOiCStKAUTjQWU9hQBMDslBAA9h2r\ncWU0IYQQ4qIpisLcsFn09PaQVX0QAFt/g5DAC1ik+qA1D2t7HXPCZhDoHTAqWYUQYiKSAu08ropZ\nCsCmUx8CEBXiR4jRh8Mnauno6nFlNCGEEOKizQ6dgUpROac5Njpb7A9viqPD4WBryXYUFK6MWjhq\nOYUQYiKSAu08YgOjSDUlUdBwghONxc5mIV09vRw6IdMchRBCeKZAb3/SglIoa66grPm0cwTNMMwR\ntOMNJyhtrmCqOZ0QvXk0owohxIQjBdoQVsb2jaJ90D+KNjvVAsC+o9UuyySEEEJcqsu/0CzENjCC\nNswmIVtLPgZgefSi0YgmhBATmhRoQ4gLjCHZmMDR+gKKbSVMMvsxKdiX3JN1tHXINEchhBCeKS0o\nBX8vP7KqDtDQ0tcsJHAYTUJKm8o51lBIkjGB6IDI0Y4phBATjhRowzBwLdr7/aNos1It9Ngd5BRa\nXRlLCCGEuGhqlZrLQmfS1tNOSUffkjLDaRKytfRjAJZHLRrFdEIIMXFJgTYMicY4Eg1x5Ncdo6Sp\njNmpfd0cs6SboxBCCA92WdhMAGqUAmDoJiHWtjpyanKJ9AsftMi1EEKIkSMF2jB9cRQt1KQnyuJH\nfnE9Le3dLk4mhBBCXJxQXwtxgTF0eFWh9+9Cqzn/24JtZZ/gwMHS6EUoijJGKYUQYmKRAm2Ykozx\nxAXGkFt7hLLmCmalWrD3OjhQINMchRBCeK65YbNAAe+QyvMe19TVzJ7K/QTpTGSap4xROiGEmHik\nQBsmRVFYGfN5R8dZA9McpZujEEIID5YRlIbDrqY7sJReR+85j9tetpOe3h6WRi1ErVKPYUIhhJhY\npEC7ACmmRGICojhozaNb00hsmD9HSxppautydTQhhBDionR0KNjrQ7GrWyloKDrrMe09Heyo2I2/\n1s953ZoQQojRIQXaBVAUhatirgT6rkWbnRpCr8NB9nGZ5iiEEMIz2Vq6sFsjgL410c7ms9N7ae/p\nYFHkPLzU2rGMJ4QQE44UaBcoLSiFKP8IDtbkEhXdt23fEZnmKIQQwjM1tnTR22LATzFw0JpHW3fb\noP3dvT18VLoDb7UXCybNdVFKIYSYOKRAu0CKorAydikOHOyp3UlCRCAFZY00NHe6OpoQQghxwWyt\nnYBCkt8Uenp7yKo+OGh/VlUOtq4m5oXPQa/VuyakEEJMIFKgXYT0oFQi/MLJrj5EaqIGB7D/uKyJ\nJoQQwvPYWvquo55mmoZKUbH79D7nvl5HL9tKP0atqFkSOd9VEYUQYkLRuDqAJ1IUhatil/Ji7p+p\n1eWhEMaOQ6e5cnoEKpWsCyOEEONJe3s7Dz/8MHV1dXR2dnLXXXexefNm8vPzMRgMANx2220sWrRo\n0HlPPvkkhw4dQlEU1q5dS0ZGhgvSD61vBA3CDSbSW1M5XJtPWXMFkf6TOFx7hOo2K5eFzcSoM7g4\nqRBCTAxSoF2kjODJhPuGcqjuMJnpcRzIa2VnbiULpoa7OpoQQogRtH37dtLT07n99tupqKjg1ltv\nJTMzkx/+8IcsXrz4rOfs27ePkpISNmzYQFFREWvXrmXDhg1jnHx4GvtH0AJ9vZkbNpPDtfnsrswi\nwi+crSUfA7AsaqELEwohxMQiUxwvkkpRcVX/tWg+USV4aVW89UkRbR09ro4mhBBiBK1cuZLbb78d\ngMrKSkJCQoY8Z/fu3Sxd2rd2Znx8PDabjZaWllHNebFsLV14aVT4eKtJC0ohwMufrKocjtUXcqqp\nlIzgNEJ9h/6ahRBCjAwp0C7BNHM6ob4hHKw7yOI5Rpraunl31ylXxxJCCDEKVq9ezY9//GPWrl0L\nwF/+8hduueUW7r//furr6wcdW1tbi9FodH5uMpmwWt1zSRZbayeBfl4oioJapWZO6Azaetp57cgb\nACyLXuTagEIIMcHIFMdLoFJUXBW9hFePvEFzQC6mgFi27i9j4bRwQkzS6UoIIcaTN998k6NHj/LA\nAw+wdu1aDAYDqamp/PGPf+R3v/sd69atO+e5DodjWM9hNOrRaNSXnNVs9h/WcfZeB01t3SRHGZ3n\nXO29kK2lH9PS3UqqOZE5CemXnGe4hpvbnXhiZvDM3J6YGTwztydmBs/N/d+kQLtE00Om8mHZDnJq\nD5MyXUXOxyFs+OgEP/i6e14MLoQQ4sLk5eURFBREWFgYqamp2O12kpKSCAoKAmDJkiU88cQTg86x\nWCzU1tY6P6+pqcFsNg/5XA0NbUMeMxSz2R+rtXlYx9pau+jtdaD3VjvP0eJLXGAMJ22nWBR2xbAf\n61JdSG534YmZwTNze2Jm8MzcnpgZPC/3+YpJmeJ4iVSKirun3ka4byjH2g5iTivm4Akr+cX1Q58s\nhBDC7e3fv59XXnkF6Ju62NbWxrp16ygrKwNg7969JCYmDjpn3rx5bN68GYD8/HwsFgt+fn5jG3wY\nbC19HRwNvt6Dtq9JuZ5vJH+NtKAUV8QSQogJTUbQRoCfly8/yPwez+X8kdMUoI3q5G8f+vLTW2ej\nVkkNLIQQnmz16tU8+uijrFmzho6ODtatW4der+e+++7Dx8cHvV7PU089BcD999/PU089xfTp00lL\nS2P16tUoisLjjz/u4q/i7Gyt/R0c/bwGbQ/zDSFMGoMIIYRLSIE2Qvy9/PhB5vf435w/UBVaQm0V\nbD8wiaUzI10dTQghxCXQ6XQ8++yzZ2z/5z//eca2X//61877P/7xj0c110ho7B9BC/T1GuJIIYQQ\nY0WGd0aQv5cf92Z+D7POjCa0hH8VvUdzW5erYwkhhBBnZRtYA83Pe4gjhRBCjBUp0EZYgJc/98/4\nPn6KEcwneW7XhmF37xJCCCHG0kCBZvCTETQhhHAXwyrQ1q9fz6pVq7j++uvZsmXLoH179uzhxhtv\nZPXq1TzyyCP09vaOSlBPEujtz4Nz7kTV5cdpVS5/zXtHijQhhBBux9baP8VRRtCEEMJtDFmg7dmz\nh8LCQjZs2MBLL73Ek08+OWj/unXreO6553jzzTdpbW1lx44doxbWkwTpDXwj5hZ62/Xstn7GOyc/\nkCJNCCGEW2ls7UJRwN9H6+ooQggh+g3ZJGTWrFlkZPSt6RUQEEB7ezt2ux21um8hzbfeesvZOthk\nMtHQ0DCKcT3L3ORoduQu55SyhS0l2/Hz1bEkZBGKorg6mhBCCIGtpZMAXy9UKvm9JIQQ7mLIETS1\nWo1erwdg48aNLFiwwFmcAc7irKamhs8++4yFCxeOUlTPoygKNy2eSvexOai6fHnryPtsKt7q6lhC\nCCEEDocDW0vXGWugCSGEcK1ht9nftm0bGzdudC7W+UV1dXV8//vf5/HHH8doNJ73cYxGPRqN+rzH\nDMf5Vt92J2azPytnp/DuXgfmmQfZdGobvr7e3JB+jaujXRBP+X5/kSdmBs/M7YmZwTNze2Jm4Z46\nuux09fSesQaaEEII1xpWgbZjxw5eeOEFXnrpJfz9B785aGlp4fbbb+e+++7jiiuuGPKxGhraLi7p\nF5jN/litzZf8OGNl+YwItu8voyVvBsZpB/hH/nu0t3VzVexSV0cbFk/7foNnZgbPzO2JmcEzc3tq\nZuGeZA00IYRwT0NOcWxubmb9+vX84Q9/wGAwnLH/F7/4Bd/61rdYsGDBqAQcD/x8tFw3P472Zi2R\nTcsI0hl5t3gLH5z60NXRhBBCTFCyBpoQQrinIUfQNm3aRENDA/fdd59z25w5c0hOTuaKK67g7bff\npqSkhI0bNwJwzTXXsGrVqtFL7KEWZYazI7eSfYeauHfNTfyj/C/85+RmVKhYHrPY1fGEEEJMMI39\nLfZlDTQhhHAvQxZoq1atOm/BlZeXN6KBxiu1SsV3v5zOuj/uZtMOKz/46vf4Tc4f+PfJ91EUhWXR\ni1wdUQghxATSNDCCJk1ChBDCrQxroWoxMjKTLUxLCKagrJGSUjv3Tb8Do7eBt4s2sa30E1fHE0II\nMYE0tg5McZQRNCGEcCdSoI2xVUsSUKsUNnx0ggCNgXsz78DgHci/TrzHh6WfujqeEEKICcLW3yTE\nIE1ChBDCrUiBNsZCTHqWzYqkrqmDzVllmPVBziLtrRPvsrXkY3odva6OKYQQYpxrbJERNCGEcEdS\noLnAtZfHEKDX8t7uUzQ0d2LRB3Nv5vcI9Arg7aJN/Cr7eUqby10dUwghxDjW1NqFr06DdgTWJhVC\nCDFypEBzAR9vDV9bGE9Xdy8bPy4CwKI388DMe5huyaC4qYT1Wb/ljeNv0dLd6uK0QgghxqPGlk4C\nZHqjEEK4HSnQXOSKKWFEhfixO7+KotM2AIw6A7el38QPpn2PEF8LOyv28NPdz7CjYo9MexRCCDFi\nunt6ae3owSBroAkhhNuRAs1FVCqFNUuTAHhjWyG9DodzX7IpgbWz7uNrCddgd9h58/hbrN//W07a\nSlwVVwghxDhi618DTa4/E0II9yMFmgslRRqYlWLh5Okm9uZXD9qnVqm5MmoB6y57gNmh0ylrruDZ\n7N/z+pG/09TV7KLEQgghxgNbf4t9g6yBJoQQbkcKNBe7YXE8Wo2Kf3x8go6unjP2B3oH8K3Jq7l/\n+p1M8gtjT9V+frL7GbaX7cTea3dBYiGEEJ7O1t/BUa5BE0II9yMFmosFB/qwYnYUjS1dbNpTes7j\nEgyxPDzrXlYlXYeiKGwsfIdfZP2GgoaiMUwrhBBiPHCugSZTHIUQwu1IgeYGVl4WjdHfm837Sqlt\nbD/ncSpFxYKIy3n8sgeYFz6bytZqfpPzB17J+yuNnbYxTCyEEMKTfb4GmkxxFEIId6NxdQAB3l5q\nvr4onhf/c4QN209w13XpKIpyzuP9vfxYk/J15oXPYUPB22TXHCK37igrY5ayOPIKNCr5ZxVCiJHS\n3t7Oww8/TF1dHZ2dndx1112kpKTwyCOP0NPTg0aj4ZlnnsFsNjvP2bt3L/feey+JiYkAJCUl8dhj\nj7nqSziD8xo0GUETQgi3I+/k3cRlk0PYfqCC7ONW3t1dwrWXxwx5TnRAJD+ecTd7Kvfz76L3ebto\nE7sq93Fj4nWkBiWNfmghhJgAtm/fTnp6OrfffjsVFRXceuutTJs2jRtvvJGVK1fy17/+lVdffZUH\nH3xw0HmzZ8/mueeec1Hq8xuY4hgo16AJIYTbkQLNTSiKwp3XpfPk6/v516cnCfT1YsHU8CHPUykq\nLg+fzTRzOu8Wb+HT8t387tBLZASncU3ccib5hY1BeiGEGL9WrlzpvF9ZWUlISAiPP/443t590wON\nRiP5+fmuindRGlu70GpU+HjL2wAhhHA38srsRoz+3vxw1TSe+ssB/vTBMQL0XkxLDB7WuXqtnhuT\nruPysNn8veBtDtfmc7g2n4zgNFbELCE6IHKU0wshxPi2evVqqqqqeOGFF9Dr9QDY7Xb+9re/cffd\nd59x/IkTJ/j+97+PzWbjnnvuYd68eWMd+ZxsLZ0E+nqddzq9EEII11Acji+skDwGrNZLX8PLbPYf\nkccZa8PNXVRh45k3cnAAD6zOJCEi8IKex+FwkF93jA9OfUhxU19nyMmmZFbEXEm8IWbUcrsTT8wM\nnpnbEzODZ+b21MzjydGjR3nwwQd555136O3t5cEHHyQ2NpZ77rln0HHV1dVkZ2dz1VVXUVZWxi23\n3MKWLVvw8jr/lMKeHjsajXo0vwR6ex189aH/kBxlZP3/zB/V5xJCCHHhZATNDcVPCuTO69L57T9z\n+c3GQzx80wwmBfsO+3xFUUgPTiUtKIWChiI+OPUhR+qPc6T+OImGOFbEXEmyMUH+ciqEEMOQl5dH\nUFAQYWFhpKamYrfbqa+v5+mnnyY6OvqM4gwgJCTEOTUyKiqK4OBgqquriYw8/2yGhoa2S847VCHf\n1NpFb68DvZfarQp+T/0DhKdlBs/M7YmZwTNze2Jm8Lzc5/sDprTZd1NTE4L59lUptHb08KsNB6lv\n6rjgx1AUhWRTAvdOv4MfTr+LyUHJFDae5LcHX+TZ7N+TV3uUMR5AFUIIj7N//35eeeUVAGpra2lr\na+Ozzz5Dq9Xygx/84KznvPPOO7z88ssAWK1W6urqCAkJGbPM59M40CBEOjgKIYRbkgLNjV2REcb1\nC+NoaO7k138/RGtH90U/Vrwhhrun3sZDM3/A1OA0iptKef7wqzyd9RsO1uTS6+gdweRCCDF+rF69\nmvr6etasWcP3vvc91q1bx9/+9jeOHDnCzTffzM0338wTTzwBwP33309HRwdLliwhKyuLNWvWcNdd\nd/HEE08MOb1xrDS1yhpoQgjhzmSKo5tbeVk0tpYutmWX89zGw/xo1TS8tBd/fUJUQATfy/gWFS2V\nbD71EQdqDvNi3uuE+oawInoJ0y0ZqFWje/2DEEJ4Ep1Ox7PPPjto25IlS8567K9//Wvn/RdeeGFU\nc10s5yLV0mJfCCHckoyguTlFUVi9NJHZqRYKy2384Z187L2XPto1yS+MW9O/yWNzfsRloTOpabPy\n2pE3+NneX7LrdBY9vT0jkF4IIYS7sbX2TXGURaqFEMI9SYHmAVSKwm1XTyY12khOYS2vby4YsWvH\nQnwt3Dz5Rh6/7EGumHQZDR2N/PXYP3hi93o+Ld9Ft/3ip1UKIYRwP5+PoMkURyGEcEdSoHkIrUbF\nPV+bQlSIH58eOs2/dxaP6OMH+5j4RvLX+MnlD7M44gpaulvZUPA2j+/+Bf85to2Ons4RfT4hhBCu\nYeu/Bk1G0IQQwj1JgeZBfLw13H/DVIIDdbzz2Sm251SM+HMYvAP5etKX+enlD7MsahEd9k5eP/RP\n1u1+iveLP6Stu33En1MIIcTYsbV0oijgr5cCTQgh3JEUaB4m0M+bH62ehr9ey182Hyf7eM2oPE+A\nlz/XJazkZ5ev5Ya0q3E4HLxbvJnHdj3Fv4vep7mrZVSeVwghxOiytXQRoPdCpZK1MIUQwh1JgeaB\nQox67rthKl5aNX945wjHSxtG7bl8tXpuSL+Gn13+CNfFr0Sr0rClZDuP7XqKjYXv0NhpG7XnFkII\nMbIcDgeNrZ2yBpoQQrgxKdA8VGxYAHd/LR2Hw8Fz/8ylvGZ0R7R0Gh3Lohfx08sf4Yakr+Cn9WV7\n2U4e3/UL/nbsn9S2143q8wshhLh0HV12urp7McgaaEII4bakQPNg6bFB3Hp1Ku2dPTz794PUNo7+\n9WFeai2LIubxxNwH+WbK1zHqDHx2ei8/2fMMr+W/SWVr9ahnEEIIcXEGGoQEyBpoQgjhtmShag83\nNy2UptYuNnx0gmf/foi1N00fkwu/NSoNl4fPZk7oDHJqDrO5ZDtZ1QfYX53DVHM6K2KWEOk/adRz\nCCGEGD5bi6yBJoQQ7k4KtHHgS7OjsLV08cG+Uv73H4d58BuZeHupx+S51So1M0MzmR4yldzaI3xw\n6iMOWnM5aM0lLSiFFTFLiAuMGZMsQgghzk/WQBNCCPcnBdo48fXF8dhaO9mdX83/vZ3H/1w/BY16\n7GawqhQVU83pZASncay+kA9KPiS/7hj5dcdINMSxIuZKko0JKIp0DRNCCFeRNdCEEML9SYE2TqgU\nhe+sTKW5rZvck3W89v4xvn1VypgWaQCKopAalERqUBInGov54NSHHK0voPDgSXRqb0L0FkJ9LYTq\nLYT4WgjztRCkM6FWjc2InxBCTGQDUxxlBE0IIdzXsAq09evXk52dTU9PD3fccQfLly937tu1axe/\n+tWvUKvVLFiw9ksGIgAAIABJREFUgLvvvnvUworz06hV3PXVdJ55I4ddeVUUVzZx07IkUmNMLsmT\nYIjlnmnfpaSpjE/Kd1HWXEFFy2lKmssG51bUWPRmQvoLt1BfC2G+IVh8gtGqtS7JLoQQ45FziqOM\noAkhhNsaskDbs2cPhYWFbNiwgYaGBr761a8OKtD+3//7f7z88suEhIRw00038aUvfYmEhIRRDS3O\nTeel4UerprHxk5N8klPBM28eZHaqhVVLEjH6u+YvptEBkdwyeRUA9l47dR31VLbWUN1aQ1VbDVWt\nNVS1VXO6tWrQeQoKQT4mZ9HWdxtCuF8o3mp5cyGEEBfK1ipNQoQQwt0NWaDNmjWLjIwMAAICAmhv\nb8dut6NWqykrKyMwMJCwsDAAFi5cyO7du6VAczG9TsstX0pmfkYYf9lSwL6jNRwqquMr82JZOjNi\nzKc9fpFa1TdaZtGbwZzm3O5wOGjstPUXazVUtVY7i7e8uqPk1R11HqtSVET7R5BgiCPBEEu8IQYf\njY8rvhwhhPAottYu9N4atBqZVi6EEO5qyAJNrVaj1+sB2LhxIwsWLECt7ntht1qtmEyfT58zmUyU\nlZWd9XHE2IsNC+DRW2aw83AlGz8u4u/bT7Azt5JvLksiNdro6niDKIqCUWfAqDOQGpQ0aF9LV+vn\nRVtrDcVNpZQ0l1PcVMrW0o9RUIjwDyfRWbDFYsbfRV+JEEK4L1tLl0xvFEIINzfsJiHbtm1j48aN\nvPLKK5f0hEajHs0I/OXObPbMN+CuyH390gCWXx7L65uO8sGeUzzzRg4Lpk3i1i+nERQ4vJEnV36/\nzfgTS+igbR3dHRTUFXPEWsBR6wkK605R1lzBR2U7AIgMDGeyOZFUcyKTzQkYfAJdEf2ieOLPtidm\nBs/M7YmZhXvosffS0t5NhNnX1VGEEEKcx7AKtB07dvDCCy/w0ksv4e//+ZsDi8VCbW2t8/Pq6mos\nFst5H6uhoe0io37ObPbHam2+5McZa67OfcPCOGYmBfOXLQV8erCCvUeqhjXt0dW5zyVMHUFYaARX\nhi6hy95NSVMphY0nOdFYTHFTCWW202w+8QkAFn1w/whbHImGOIw6g4vTn527fq/PxxMzg2fm9tTM\nwj3YWgZa7EsHRyGEcGdDFmjNzc2sX7+e1157DYNh8JvaiIgIWlpaKC8vJzQ0lO3bt/PLX/5y1MKK\nS+dJ0x4vhJdaS6IxnkRjPABGkw/Zxcc40XCSQttJTjae4rPT+/js9D4AgnRGko0JTLNMIdmYgEYl\nK04IIca3gTXQZIqjEEK4tyHflW7atImGhgbuu+8+57Y5c+aQnJzMsmXLeOKJJ/jRj34EwMqVK4mN\njR29tGJEqBSFBVPDmZ5k5q1P+7s9vpHj8m6PI0mj1hAXGE1cYDTLWYy9105FS6VzhO1E40l2VWax\nqzILH40PGcGTybRMIcWUhFaKNSHEOCRroAkhhGcY8p3oqlWrWLVq1Tn3z5o1iw0bNoxoKDE2/Hzc\nt9vjSFOr1EQFRBAVEMGVUQvodfRSbCslx3qYnJpc9lZls7cqG51ax5TgVDItU0g1JeMl67AJIcaJ\nRhlBE0IIjyBDBWLcTns8H5WiIt4QQ7whhq8lXENJUzk51sMcrMklqzqHrOocvNVepAelkmnJIC0o\nGS9Ze00I4cEGRtAMvvJaJoQQ7kwKNAGcf9rjPasyXR1vVKkUFbGBUcQGRvHV+KspbS4npyaXnJrD\nZNccIrvmEF4qLWlBKWRappAWlIpOI1OEhJgo2tvbefjhh6mrq6Ozs5O77rqLlJQUHnzwQex2O2az\nmWeeeQYvr8GFz5NPPsmhQ4dQFIW1a9c61xR1lc+vQZPXLyGEcGdSoIlBzjbt8Z5ntnPz8mRmJJtd\nHW/UKYpCdEAk0QGRfCX+KspbKjlYc5gD1sPkWHPJseaiVWmYHJRCpnkK6cGp+Gh0ro4thBhF27dv\nJz09ndtvv52KigpuvfVWpk+fzpo1a7jqqqv41a9+xcaNG1mzZo3znH379lFSUsKGDRsoKipi7dq1\nLr8cYKCLo0xxFEII9yYFmjirgWmPH+4v55+fFPH7f+VyeXooa5YmoddNjB8bRVGI9A8n0j+ca+K+\nxOnWqr6RNWsuh6x5HLLmoVHUxBti8dXq8VJ54aXWolVrnfc/v9WiVZ+5zUvt5TxempMI4Z5Wrlzp\nvF9ZWUlISAh79+7lJz/5CQCLFy/mlVdeGVSg7d69m6VLlwIQHx+PzWajpaUFPz+/sQ3/BY0tnWjU\nKvTe8lojhBDuTF6lxTmpFIVlsyKZPyOS9X/OYldeFcdKG7h1ZSqTY0yujjemFEVhkl8Yk/zCuCZu\nOZWt1RzsL9aON5wYmedAIVDnT4DWn0DvQAzegRi8A/rvBzg/16l1KIoyIs8phBi+1atXU1VVxQsv\nvMB3vvMd55TGoKAgrFbroGNra2tJS0tzfm4ymbBarS4t0GytXRj8vOT1Qwgh3JwUaGJIkSH+rL15\nBu/tLuE/n53il28eZOmMCK5fFI+3Vu3qeC4R5htCWGwIV8UupaOnk67eLrrs3XTZu+ju7bvt6u12\nbuvq7ab7i9ucx3fT3X+/w95Ja08Lp1urKW2uOOdze6m9+go3r4GiLZDALxRwA5+rlPHThVMId/Dm\nm29y9OhRHnjgARwOh3P7F++fy3COATAa9Wg0l/66+t8LhPf2Omhq7SIx0uDWi4e7c7Zz8cTM4Jm5\nPTEzeGZuT8wMnpv7v0mBJoZFo1bxlStiyYgP4qV3j7Atu5y84npuv3YysWEBro7nUjqNNzpG5qJ7\ns9mfmpomWnvasHU20djZhK3TRmOn7Qv3m2jstFHTVnvOx9GoNFh8grHozYT0fwzc12t9RiSrEBNF\nXl4eQUFBhIWFkZqait1ux9fXl46ODnQ6HdXV1VgslkHnWCwWams//z9aU1OD2Tz0dbwNDW2XnNds\n9sdqbR60ramtC3uvA7235ox97uJsud2dJ2YGz8ztiZnBM3N7YmbwvNznKyalQBMXJDYsgMe/PYt/\nfnKSrfvL+Pmfs7nm8miuuTxmXK2b5kqKouCn9cVP68skv7BzHtfT24Otsxlb1+dFW2OnjcYOG9b2\nOmrarJxurTrjPH+tn7NYs+iDnQVcsE8QapX7joh29/bQ09sjTVnEmNu/fz8VFRU8+uij1NbW0tbW\nxvz589m8eTNf+cpX2LJlC/Pnzx90zrx58/jtb3/L6tWryc/Px2KxuHZ6ozQIEUIIjyEFmrhgXlo1\n31iayLSEIF7edJR3PjvFoaI6br9mMuHBvq6ON2FoVBqCfIwE+Zx9rTqHw0FTVzPVbVaq26zU9N9W\nt1k5aTtFka140PEqRUWwj2nQaJvR24CPxge91ge9xgcfjQ7NKDQz6ejpoLHTRsPACGGHjcauvmJz\noPBs6W5FQWGaZQrLohYSHRA54jmEOJvVq1fz6KOPsmbNGjo6Oli3bh3p6ek89NBDbNiwgfDwcK67\n7joA7r//fp566immT59OWloaq1evRlEUHn/8cZd+DbIGmhBCeA4p0MRFS40x8dNb5/DGtgI+y6vi\niVez+PqieJbOjEAlF6G7nKIoBHoHEOgdQJIxftC+7t4erG21g4q2GudtLXD0nI/rpdKi1+rx0ejw\n0fhg8PVHbdf2F3F92/QaH3ycRZ0P4HBO0xwY5XOO+HXa6LB3nvf5DLpAwv3CaO1uJafmMDk1h0k0\nxLE0aiFpQSnS9ECMKp1Ox7PPPnvG9ldfffWMbb/+9a+d93/84x+Paq4LIWugCSGE55ACTVwSvU7D\nbddMZlqimT99cIw3PyzkYKGVW69OJThQrnVyV1qVhnC/UML9QgdtdzgctHS3Ogu2pq5m2nraae/u\n6Lvtae//vB1bZxNVrTU4bMNrfnA2vho9Jp0Rgy4Qo3cggd59t4YvfPhoPu9a6XA4ON5wgm2ln3C0\nvoDCxpOE+4ZyZdQCZoZMG5XRPSHGg8b+EbRAGUETQgi3J+9mxIiYkWwmISKQP71/jIMnaln38j7W\nLE1i3pRQGd3wIIqi4O/lh7+XHwmG2CGP73X04m/worTK+nnx1tNOW/fnxVxbTwfgwOAViEEXOKjb\npJf6wt4sKopCiimRFFMi5c2n2Vb6Cdk1h3j96N/5z8nNLI68gnnhc+Q6NSH+y8A1aAYZQRNCCLcn\nBZoYMYG+XvzP9VPYmVvJG9sKeWXTUXIKrXxrRQoB8lfbcUmlqNB7+fRfB3f2a+FGS4R/ON9O+wbX\nxq1ge/kOPju9j3+deI/3iz9k/qTLWBQ5D4N34JhmEsJdNbZKkxAhhPAUUqCJEaUoCvMzwkmNMvLy\ne0fJKazlRMVevr0ihcykoVtMC3GhgnyMfD3xy6yMWcqnFXv4uHwnW0s/5qOyHcwKzWRp1ELCfENc\nHVMIl2pq6URRIEAvBZoQQrg7KdDEqAg2+PDAmky2ZZWx8ZOT/PatXNLjTCybGUlarEmaiIgRp9fq\nWRGzhCsj57Ov6gDbyj5hT+V+9lTuJz0olWXRi4gPjJEpt2JCamztwl/vhUolP/9CCOHupEATo0al\nKCyfHUVaXBCvbz5O3sl68k7WE2rSc+WMCOZNCUXnJT+CYmRp1VrmTZrD3PBZ5NYeYWvJJ+TVHSWv\n7iixAVEsjVpIhjnN1TGFGFO2li5CjNK4SQghPIG8OxajblKwLw9/czqnqprYtr+cfUer+evWAt76\n9CTzM8K4ckYEZoO8cRAjS6WomGpOZ6o5naLGU2wr/YTDtfm8mPc6QToTIf5BdHfbUVBAUVCg7z44\nR9n69vXdOrcoA3v6tjkAB704HODAgcPhOPut8z5n2ebAW+1FoFeAc2kEwxfuB3gFoNNIcwdxcdo7\ne+jstkuLfSGE8BBSoIkxExMawHevmcwNixP4OKeC7TkVbMkqY2tWGdMSg1k6M5KUKINMQRMjLt4Q\nQ7whhqrWGj4s/ZSs6gPUWevHPIeCgqIoX7jFWSB227vpK9/OTqfWYdIH4qf2cxZugd4BzqLO0F/I\neam1Y/cFCY/QNNAgRJo1CSGER5ACTYy5QF8vvnJFLFfPjSbraA1b95eRU1hLTmEtEWY/ls6M4LLJ\nIXhp1a6OKsaZUF8L30z9Ot9M/TrBwX7UWJuAvhEt6B8B69vA51sG9n9hm+PzfUr/qFrfrcpZdA0u\nxJQh//Bg77XT1NWMrasJW2czts6m/vtNzvvNXc2c7qw+7+Po1N7oNLr+hcR16NT9t+fZptN446P2\nce5Xq+T/3njiXANNOjgKIYRHkAJNuIxGrWJueiiXpYVQdLqJbfvL2H/MymvvH2Pjx0UsnBbO4sxJ\nmAJkTSsx8hRFQaWo+j9xbRYAtUqNUWfAqDOc8xiz2Z/K6oa+Qq6/cGvsL+KaOpv7i7gW2ns6aOps\nprrNSq+j94KzaFVagn1M3Jf5ffy8fC/lyxJuwNYqa6AJIYQnkQJNuJyiKCRMCiRhUiD1izvYnlPB\nJwdP897uEt7fU8qMZDPLZkYSPylApj+KCU+j0mDSGTHphl53zuFw0NXbTUdPB+39Hx09HbTbO2jv\naXd+3tHT2bffPnBMO1qV1+cFrPBojS0yxVEIITyJFGjCrZgCdFy/MJ5rL49hz5Fqtu0vI+tYDVnH\naogJ9WfpzAhmpYSg1cgbRyGGoigK3mqvvgYk3gGujiNcxNYqUxyFEMKTSIEm3JKXVs2CqeHMzwjj\neGkjW/eXcbCwlpfePcrrWwpIjTIyJc5EWlwQFukAKYQQ52QbGEGTKY5CCOERpEATbk1RFFKijaRE\nG7E2trM9p4KDhbUcPNH3ARBi9CE9Loj0WBMpUUa8vaTBgRBCDLD1NwkxyBRHIYTwCFKgCY9hNvhw\n4+IEblycgLWxnbzievJO1nGkpIEPs8v5MLscjVohKdLAnPRwYkN8mRTsK9etCSEmtMbWLny8NdIZ\nVwghPIQUaMIjmQ0+LM6cxOLMSfTYeymqsJF7sp684jqOnGrgyKkGAIz+3qTFmpgSF8TkGCO+Olkj\nSggxsdhauqRBiBBCeBAp0ITH06hVJEcZSY4y8vVF8dhaOimta2f3oQryiuvZebiSnYcrURSICw9g\nSmwQ6XFBxIT6o1LJ6JoQYvzqsffS0t5NhFmWSxBCCE8hBZoYdwL9vFkSG8yUaAO9vQ5KqpvJO1lH\nbnE9RRU2iiqaeHtnMf56LdMSgpmRbCY12iSdIYUQ405TqzQIEUIITyMFmhjXVCqF2LAAYsMCuHZe\nLG0d3Rw51UDuyToOFdWx43AlOw5XovNSMzUhmBlJZtLjTOi85L+GEMLzyRpoQgjheeRdqJhQ9Dot\nM1MszEyx0NvroOi0jezjVg4UWNl7pJq9R6rRalSkxZiYkWxmakIwfj5y3ZoQwjPJGmhCCOF5pEAT\nE5ZKpZAYYSAxwsCqJQmU1bQ4i7WBNv4qRSE5ysCMZDOZiWaM/jJNSAjhOQbWQDP4ymuXEEJ4CinQ\nhKBvvbWoEH+iQvz56oI4qurbOFBgJfu4laMlDRwtaeAvWwqInxTA9CQzM5LMWIx6V8cWQojzamyR\nETQhhPA0UqAJcRahJj0rL4tm5WXR1Dd1cKCgb2TteFkjRRVN/GN7ERFmv/6RtWAiLX6y3poQ49z6\n9evJzs6mp6eHO+64g3fffZeGhr4lPRobG5k2bRo/+9nPnMe/9dZb/OY3vyEqKgqAyy+/nDvvvHNM\nM9ukSYgQQnicYRVoBQUF3HXXXXz729/mpptuGrTvr3/9K++88w4qlYr09HQeffTRUQkqhKuYAnQs\nnRnJ0pmRNLV1caiwluwCK0dO1fPvnS38e2cxgX5epMWYSI81MTnGRIBckC/EuLJnzx4KCwvZsGED\nDQ0NfPWrX+Xjjz927n/kkUe44YYbzjhv5cqVPPTQQ2OYdDCbNAkRQgiPM2SB1tbWxs9+9jPmzp17\nxr6WlhZefvlltmzZgkaj4dZbb+XgwYNMmzZtVMIK4WoBei/mTw1n/tRw2jt7yD1Zx8HCWvJP1bMr\nr4pdeVUARIX4kRZrIj3GREKEQVr4C+HhZs2aRUZGBgABAQG0t7djt9tRq9WcPHmS5uZm5353Ymvt\nRKNW4auTCTNCCOEphnzF9vLy4sUXX+TFF188Y59Wq0Wr1dLW1oZer6e9vZ3AwMBRCSqEu/Hx1jA7\nNYTZqSH0OhyU17SQX1xPXnE9heWNlFa38P6eUrw0fQtpp8WaSIs1ER6kl+mQQngYtVqNXt933enG\njRtZsGABarUagD//+c9nzC4ZsG/fPm677TZ6enp46KGHmDx58phlhr42+4G+XvKaI4QQHmTIAk2j\n0aDRnP0wb29v7r77bpYuXYq3tzdXX301sbGxIx5SCHen+kKTkasui6azy87xskbyi+vJP1VP7sk6\nck/WAWD09yYtpq9YmxxjxF8vU4+E8BTbtm1j48aNvPLKKwB0dXWRnZ3NE088ccaxU6dOxWQysWjR\nInJycnjooYf4z3/+c97HNxr1aDTqS85pNvvT2+ugqbWLhEgDZrP/JT/mWPCUnF/kiZnBM3N7Ymbw\nzNyemBk8N/d/u6Q5Dy0tLfzhD3/ggw8+wM/Pj29961scO3aMlJSUc54zkr98PJHkHjuuzhwxycCV\nl8UAUNvYzsGCGnKOW8kpsLIzt5KduZUoCsRPCiQz2UJmkgWjydfluS+GJ2YGz8ztiZnHix07dvDC\nCy/w0ksv4e/f9++QlZV1zqmN8fHxxMfHA5CZmUl9fb1zWuS5NDS0XXJOs9kfq7WZ5rYu7L0O9F5q\nrNbmS37c0TaQ25N4YmbwzNyemBk8M7cnZgbPy32+3+eXVKAVFRURGRmJyWQCYObMmeTl5Z23QBvJ\nXz6eRnKPHXfMPDXWxNRYE7d8KYnS6ua+0bXiegrLbZwot/GPDwvx89EyJS6I6Ulm0uNMeGsv/Y8Z\no80dv9fD4Ym5PTXzeNDc3Mz69et57bXXMBgMzu25ubnn/J334osvEhYWxjXXXENBQQEmk+m8xdlI\nc66BJh0chRDCo1xSgTZp0iSKioro6OhAp9ORl5fHwoULRyqbEOOSSlGICQ0gJjSAq+fG0NHVw/HS\nRg6frONwUR2786vYnV+Fl0ZFWqyJ6UlmpiYE4+ejdXV0ISasTZs20dDQwH333efc9vTTT2O1Wp1t\n9AfceeedPP/881x77bU88MADvPnmm/T09PDzn/98TDM3tsoaaEII4YmGLNDy8vJ4+umnqaioQKPR\nsHnzZpYsWUJERATLli3jtttu45ZbbkGtVpOZmcnMmTPHIrcQ44bOS8PUhGCmJgQTHOxHVu5p57pr\nOYW15BTWolIUkqMMTE/qW3fNFKBzdWwhJpRVq1axatWqM7Y/9thjZ2x7/vnnAQgNDeX1118f9Wzn\nIiNoQgjhmYYs0NLT08/7C2b16tWsXr16REMJMVEpikJsWACxYQFcvzCeyrrW/mKtlqMlDRwtaeCv\nWwuICfVnepKZ6UlmwoN9XR1bCOGGBhaplnUZhRDCs8jCKEK4sbAgX66e68vVc2Oob+roH1Gzcry0\nkVNVzbz16UlCTXoyk4KZnmQmNiwAlbTTFkIAjS19UxwNMsVRCCE8ihRoQngIU4COK2dEcOWMCFo7\nujl0opYDBbXknazj/T2lvL+nFIOfF5mJfQ1GkiIN+OrkujUhJqqBKY6BvjLFUQghPIkUaEJ4IF+d\nlsvTw7g8PYzObjtHius5UGDl4IlatudUsD2nAgWIDPEjJcpIcqSBpCgp2ISYSGytXShAgK/8vxdC\nCE8iBZoQHs5bqyYzyUxmkhl7by+FZTaOlTZwvLSRotM2Sqtb2JJV1lewWfxIjjKSHGUgKdIgnSGF\nGMdsLZ3467WoVSpXRxFCCHEBpEATYhxRq1SkRBtJiTYC0NVt5+TpJo6VNlBQ1siJiiZKa1rYur+v\nYIuw+JEcZSAlyigFmxDjTGNrFxaDj6tjCCGEuEBSoAkxjnlp1YMKtu6egYKtkeOlDZyoaKKspoVt\n+8sBiDD7kRJlcI6yScEmhGfq6Oqhs8sua6AJIYQHkgJNiAlEq1H3F19GINZZsB0vbeR4WSMnKmyU\nW1vYlt1XsEVZ/JgSH0R6rIn4SYFo1DJVSghPMNBi3yANQoQQwuNIgSbEBDa4YIPunl6KK5uc17AV\nljdSWtPCe7tL8PHWMDnGyJS4IKbEBWH0lzd+QrgrZwdHGUETQgiPIwWaEMJJq1GRFNnXQIR50Nll\n52hpA7kn68gtqiP7uJXs41YAIsy+TIkLYv70SIL9tDK6JoQbGVgDLVAWqRZCCI8jBZoQ4py8vdRM\nSwhmWkIwDoeD6oZ2covqyD1Zx7HSRsqtpby/txSdl5rUaCNT4oPIiAvCFKBzdXQhJrSBETSDn4x0\nCyGEp5ECTQgxLIqiEGrSE2rSs2xWJJ3ddo6XNnKisol9+VXkFNaSU1gLQHiwL1PiTEyJCyIxwoBW\nI6NrQoylgWvQZIqjEEJ4HinQhBAXxVurJiM+iCsvi+FrV8RS3dBG3sn6vtG1kgY27ytj874yvLVq\n4icFEGLSYzH49H0YfTAbfPDSql39ZQgxLtlkiqMQQngsKdCEECMixKgnZIaeK2dE0N1j53hZI7lF\nfQXbkVMNHDnVcMY5Bj8vLMa+ws1s/Lx4sxh98NVJi38hLlajcwRNpjgKIYSnkQJNCDHitBo16bFB\npMcG8Q0Sae/swdrYjrWxnZqGdmoGbhvaKSxvpKCs8YzH8NVpMH+hYDMbfAgP9iU2LACVorjgqxLC\nc9haOvHxVuMto9RCCDFsH3/8IYsWXTnkcb/5zbPccMNqwsMnjUoOKdCEEKPOx1tDVIg/USH+Z+zr\nsfdSa+voL9jaqGlsx9pfxJVbWzlV1TzoeLNBxxVTwpg3JUyakQhxDrbWLgJkDTQhhBi2ysrTbNu2\neVgF2r33/mhUs0iBJoRwKY1a5Ww+AkGD9vU6HDQ2d1Ld0Df6VljWSNbxGv61o5i3dxSTFmti/tRw\npiUESyMSIfr12HtpbusmPMjX1VGEEMJj/OpXT3P0aD7z589i+fKrqKw8zf/+7//x1FM/xWqtob29\nnVtv/R7z5s3nnnu+xw9/+CDbt39Ia2sLpaUlVFSU84Mf/Ii5c+ddchYp0IQQbkulKJgCdJgCdKRG\nG1kwNZw1y5LIOlbDjsOnySuuJ6+4Hl+dhrlpoVyREXbWUTohJpLG5v4GIdLBUQjhof7+0QmyjtVc\n0DlqtYLd7jjn/lkpFm5cknDO/d/4xs289dbfiY2Np7T0FP/3fy/R0FDP7NmXcdVV11BRUc5jjz3M\nvHnzB51XU1PNL3/5HHv27OLf//6nFGhCiInHx1vDgqnhLJgazunaVnbmVrIrt5Jt2eVsyy4nOtSf\n+RlhzJkcIo1GxIRU39QByBpoQghxsVJT0wDw9w/g6NF83nnnLRRFRVOT7YxjMzKmAWCxWGhpaRmR\n55cCTQjhscKDfblxcQJfWxBHblEdOw5Xcriojr9sKWDDRyeYkWTmiowwUqKN0lhETBjOETRpsS+E\n8FA3Lkk472jX2ZjN/litzUMfOAxabd8feLdu/YCmpiZ+//uXaGpq4rvfvfmMY9Xqz5sxORznHsG7\nEFKgCSE8nkatIjPJTGaSGVtLJ7vyq9hxqJI9R6rZc6SaoAAdV2SEMW9KKMGBPq6OK8SoGhhBkymO\nQggxfCqVCrvdPmhbY2MjYWHhqFQqPvnkI7q7u8ckixRoQohxJdDPm6vmRLNidhRFFU18evg0WUdr\n+PfOYt7ZWczkGCPzMsK4LEODw+FAkZE1Mc40OAs0meIohBDDFR0dy/HjxwgLC8dgMACwaNESHn74\nhxw5ksfVV38Zi8XCq6++OOpZpEATQoxLiqKQEBFIQkQga5YmknWshp2HK8k/1UD+qQb++M4RfLzV\nhAf7MinYl/BgPyaZ++4H+npJ4SbOsH79erKzs+np6eGOO+7go48+Ij8/3/mL/LbbbmPRokWDznny\nySc5dOiNybcXAAARUklEQVQQiqKwdu1aMjIyRj1nff8UR4NMcRRCiGEzGo289dZ7g7aFhYXzpz+9\n6fx8+fKrAPjOd24HIC7u82mYcXEJ/O53fxyRLFKgCSHGPZ2XhvkZ4czPCKeyrpX9x61YbR2crLBx\nqrKZooqmQcf76jRMCvZlktmP8GBfIsy+hP//9u49KMp6/wP4e9nltgsL7MIC6wV1jxbSTdMSTRAS\nS8/JUs+ozI8xRxrzgiZqiE4I5TnhhSwH+2Vgaql5mKhprJzRcTJ/ngI08meB5xxRy5/KRWC5LZdg\nl+f3x+qTKzdN2efZfL9mmGGf7z7Le77znf3uh+e73ydQA181P/DerwoLC1FWVoa8vDzU1dVhxowZ\nGDduHFauXImYmJhuzzl58iQuXbqEvLw8XLhwAevWrUNeXl6/Z+UVNCIi18YCjYjuK6F6DZ4brxG/\nTGy1daLS3ILymmZcqW5GeU0zrlZbUHa1AeeuOO7WpFW7i0XbjattxkANd4u8D4wdO1a8+qXVatHa\n2trluwq3KigowOTJkwEAJpMJDQ0NsFgs8PHx6desdU1tUCkV0HhxiicickV89yai+5pK6YaBQT4Y\nGOSDJ8J/O97eYUNF7fXCrcaC8upmXK1pxr8u1eFfl+ocXsPPxwNGvcb+E6iGMVCD0EANtLzi9oeh\nVCqhVqsBAPn5+YiKioJSqcS+ffuwe/du6PV6pKWlQafTiefU1NQgIiJCfKzT6VBdXd3vBZq58Vcu\n0yUicmEs0IiIuuHhrkRYiC/CQhxvfN3WbkVFbQuuVFtQXtOM8hp7Eddd4ebj7Q6j/reCzRhoL+L8\nffjh2VUdPXoU+fn52LVrF0pKSuDv74/w8HDk5ORg+/btWL9+fY/n3s72ywEBaqhUyj6f19vfqG9q\ng2mAP4KCXO+m7czsPK6Y2xUzA66Z2xUzA66b+1Ys0IiI7oCXhwpDQ7UYGqp1ON7WbhWXSt4o2spr\nm7tdKuntqYJRr7YXbfobhZsaOj8v3q9Nxk6cOIEdO3Zg586d8PX1RWRkpNgWGxuLjIwMh+cbDAbU\n1NSIj69du4agoKBe/0ZdXctdZbS0dsBqE6D2VN6z+wE5y728h5GzuGJmwDVzu2JmwDVzu2JmwPVy\n91ZMskAjIroHvDxUGBKixZAQx8Ktw2pDpbn1euFmL9oqalvwS2UTLpQ7bk7irnJDcIAaoXr7T4hO\njVC9BsE6b3h58O1aSk1NTdi8eTP27Nkj7tq4bNkypKSkYNCgQSgqKsLw4cMdzpkwYQKys7Mxd+5c\nlJaWwmAw9PvyxnrL9ZtUc4MQIiKXxRmfiKgfuauUGGTwwSCD4wdzq60T1+paxaKtsrYFFbUtqDTb\nl0/eKsDXE2EhWuh9PRGiVyNEr0aoTo0AX08ul3SCQ4cOoa6uDitWrBCPzZw5EytWrIC3tzfUajUy\nMzMBAMnJycjMzMTo0aMRERGBuXPnQqFQID09vd9zNljaAXCLfSKi/vLXvz6Hjz7KE7+X3B9YoBER\nSUCldLMvbQzUOBwXBAF1Tb+iwtxyvWhrRqXZXrz9b1l1l9fx9FAi5PpVtxC9GgMCNRhm9EOAL6+g\n3Etz5szBnDlzuhyfMWNGl2Nvv/22+Pvq1av7NdetfruCxgKNiMhVsUAjIpIRhUIBndYLOq0XIobo\nHNo0vl4oLbsmXm2zF3H23SUvVTmuuw/w9cQwoxbDjFqYjH4IC/GFp/vv33yCXENjs/0Kmp+GBToR\n0Z1YsOC/8OabbyEkJASVlRVYu3YVgoIMaG1tRVtbG5KTX8XIkQ85JQsLNCIiF6H2cu/2e26dnQJq\nG9tQUduMy9csuFjeiAvljSj+TzWK/2O/6uamUGCgwX51zXS9cAvWqbkpyR9M/fUljryCRkSu7LPz\nX+L0tZ/u6BylmwK2zp53yx1leBgz//SXHtujomLw7bf/g1mzZuPEieOIioqByTQcUVGTUFx8Cvv3\nf4i//33LHWX6vVigERG5ODc3BYL8vRHk741HTIEA7EslaxvbcLG88XrB1oBLlRb8X5UF35y+CgBQ\ne6ow1KjFsFAtTAO0GGb0g483b7rtyhqa7Usc/blJCBHRHYmKisH27e9g1qzZ+Oc/jyMpKRn/+Mde\nHDiwFx0dHfDy8nJaltsq0M6dO4clS5Zg/vz5SEhIcGirqKjAypUr0dHRgZEjR+KNN97ol6BERHT7\nFAoFAv28EejnjSfCgwHYNya5cYXtYnkDLpQ3ovRnM0p/NovnGQK8McyoxcgwHcY/HMIrbC6m3tIO\nhQLwVbPQJiLXNfNPf+n1ald37nab/WHDTKitrUZVVSWamppw4sQ3CAw0IC1tA/7977PYvv2d3/3a\nd6rPAq2lpQUbNmxwuN/LzTZu3IgFCxYgLi4Or7/+OsrLy2E0Gu95UCIiujsqpZt4D7enHx8IAGhq\nacfPFY3issifyxtRWFqFwtIqjBjsD4O/t8Sp6U50CgIMAWqolG5SRyEicjmRkU8hJ+e/MXFiNOrr\n62Ay2W+fcvz4MVitVqfl6LNA8/DwQG5uLnJzc7u0dXZ2ori4GFu3bgUAp2whTERE946v2gOPmALF\npZGdgoAqcwua26wszlzQoukR8PNXA52dUkchInI50dExWLRoAfbsOYC2tlb87W/pOHbsKGbNmo2j\nR4/gq68OOiVHnwWaSqWCStX908xmMzQaDTIzM1FaWooxY8Zg1apVvb5eQIAaKtXd7yTW29235Yy5\nnccVMwOumdsVMwOumdsZmYMN2r6fRLKk03ohSK+5q2U+RET3q/DwCBw/XiQ+3r8/X/z9qaeiAQB/\n/vP0fs9xV5uECIKAqqoqzJs3DwMGDMDChQvxzTffYNKkST2eU1fXcjd/EsDdrzGVCnM7jytmBlwz\ntytmBlwzt6tmJiIiott3V4vUAwICYDQaMXjwYCiVSkRGRqKsrOxeZSMiIiIiIrqv3FWBplKpMGjQ\nIPzyyy8AgNLSUgwdOvRe5CIiIiIiIrrv9LnEsaSkBJs2bcLVq1ehUqlw+PBhxMbGYuDAgYiLi8O6\ndeuQmpoKQRAwYsQIxMbGOiM3ERERERHRH06fBdpDDz2EvXv39tgeFhaGAwcO3NNQRERERERE9yPe\nKIWIiIiIiEgmWKARERERERHJBAs0IiIiIiIimWCBRkREREREJBMs0IiIiIiIiGRCIQiCIHUIIiIi\nIiIi4hU0IiIiIiIi2WCBRkREREREJBMs0IiIiIiIiGSCBRoREREREZFMsEAjIiIiIiKSCRZoRERE\nREREMqGSOkBf3nzzTZw5cwYKhQLr1q3DI488IrZ999132Lp1K5RKJaKiorB06VIJkzravHkziouL\nYbVa8fLLL2PKlCliW2xsLEJCQqBUKgEAWVlZCA4OlioqAKCoqAivvPIKhg8fDgAYMWIE0tLSxHa5\n9vUnn3yCgwcPio9LSkpw+vRp8XFERARGjx4tPt6zZ4/Y78527tw5LFmyBPPnz0dCQgIqKiqQkpIC\nm82GoKAgbNmyBR4eHg7n9Db+pcy9du1aWK1WqFQqbNmyBUFBQeLz+xpLUmROTU1FaWkp/P39AQCJ\niYmYNGmSwzly7Ovly5ejrq4OAFBfX4/HHnsMGzZsEJ//2WefYdu2bRg8eDAAYPz48Vi8eLHTc5M0\nOD86B+dH5+AcKV1mzpEyJMhYUVGRsHDhQkEQBOH8+fPC7NmzHdqnTp0qlJeXCzabTYiPjxfKysqk\niNlFQUGB8NJLLwmCIAhms1mIjo52aI+JiREsFosEyXpWWFgoLFu2rMd2ufb1zYqKioSMjAyHY088\n8YREaRw1NzcLCQkJwmuvvSbs3btXEARBSE1NFQ4dOiQIgiC89dZbwv79+x3O6Wv8O0N3uVNSUoSv\nvvpKEARB2Ldvn7Bp0yaHc/oaS/2tu8xr1qwRvv766x7PkWtf3yw1NVU4c+aMw7FPP/1U2Lhxo7Mi\nkoxwfnQezo/9j3Ok83COdA2yXuJYUFCAyZMnAwBMJhMaGhpgsVgAAJcvX4afnx9CQ0Ph5uaG6Oho\nFBQUSBlXNHbsWGzbtg0AoNVq0draCpvNJnGq30/OfX2zd999F0uWLJE6Rrc8PDyQm5sLg8EgHisq\nKsLTTz8NAIiJienSp72Nf2fpLnd6ejqeeeYZAEBAQADq6+udmqkv3WXui1z7+oaLFy+iqalJkv9Y\nkjxxfpQHOff1zeQ8PwKcI52Jc6RrkHWBVlNTg4CAAPGxTqdDdXU1AKC6uho6na7bNqkplUqo1WoA\nQH5+PqKiorosG0hPT0d8fDyysrIgCIIUMbs4f/48Fi1ahPj4eHz77bficTn39Q0//vgjQkNDHZYR\nAEB7eztWrVqFuXPnYvfu3RKlA1QqFby8vByOtba2iss19Hp9lz7tbfw7S3e51Wo1lEolbDYbPv74\nYzz33HNdzutpLDlDd5kBYN++fZg3bx6Sk5NhNpsd2uTa1zd89NFHSEhI6Lbt5MmTSExMxIsvvoiz\nZ8/2Z0SSEc6PzsX5sX9xjnQezpGuQfbfQbuZXN6ob9fRo0eRn5+PXbt2ORxfvnw5Jk6cCD8/Pyxd\nuhSHDx/Gs88+K1FKuyFDhiApKQlTp07F5cuXMW/ePBw5cqTLem+5ys/Px4wZM7ocT0lJwfTp06FQ\nKJCQkIAxY8bg4YcfliBh725nbMtp/NtsNqSkpGDcuHGIjIx0aJPjWHr++efh7++P8PBw5OTkYPv2\n7Vi/fn2Pz5dTX7e3t6O4uBgZGRld2h599FHodDpMmjQJp0+fxpo1a/DFF184PyRJTk5j9nZwfnQe\nV58fAc6R/Y1zpPzI+gqawWBATU2N+PjatWvif4Bubauqqrqjy7X97cSJE9ixYwdyc3Ph6+vr0PbC\nCy9Ar9dDpVIhKioK586dkyjlb4KDgzFt2jQoFAoMHjwYgYGBqKqqAiD/vgbsSyFGjRrV5Xh8fDw0\nGg3UajXGjRsni76+Qa1Wo62tDUD3fdrb+Jfa2rVrERYWhqSkpC5tvY0lqURGRiI8PByAfROCW8eB\nnPv61KlTPS7bMJlM4he5R40aBbPZ7NLLxej2cX50Hs6P0uAc6TycI+VH1gXahAkTcPjwYQBAaWkp\nDAYDfHx8AAADBw6ExWLBlStXYLVacezYMUyYMEHKuKKmpiZs3rwZ77//vrgjzs1tiYmJaG9vB2Af\nWDd28pHSwYMH8cEHHwCwL9mora0Vd86Sc18D9jdujUbT5b9PFy9exKpVqyAIAqxWK3744QdZ9PUN\n48ePF8f3kSNHMHHiRIf23sa/lA4ePAh3d3csX768x/aexpJUli1bhsuXLwOwf1i5dRzIta8B4Kef\nfsKDDz7YbVtubi6+/PJLAPbdrXQ6naS7sJHzcH50Hs6P0uAc6TycI+VHIcjpOmU3srKy8P3330Oh\nUCA9PR1nz56Fr68v4uLicOrUKWRlZQEApkyZgsTERInT2uXl5SE7OxtDhw4Vjz355JN44IEHEBcX\nhw8//BCff/45PD09MXLkSKSlpUGhUEiYGLBYLFi9ejUaGxvR0dGBpKQk1NbWyr6vAfvWwe+88w52\n7twJAMjJycHYsWMxatQobNmyBYWFhXBzc0NsbKxk26uWlJRg06ZNuHr1KlQqFYKDg5GVlYXU1FT8\n+uuvMBqNyMzMhLu7O5KTk5GZmQkvL68u47+nNyFn5q6trYWnp6f45mwymZCRkSHmtlqtXcZSdHS0\npJkTEhKQk5MDb29vqNVqZGZmQq/Xy76vs7OzkZ2djccffxzTpk0Tn7t48WK89957qKysxKuvvip+\nyJJq62OSBudH5+D86JycnCOly8w5Un5kX6ARERERERHdL2S9xJGIiIiIiOh+wgKNiIiIiIhIJlig\nERERERERyQQLNCIiIiIiIplggUZERERERCQTLNCIiIiIiIhkggUaERERERGRTLBAIyIiIiIikon/\nBzrX8YXZXcDEAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fb6b36bf438>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RAQROqCgfcQM",
        "colab_type": "code",
        "outputId": "9294984b-9af4-46e8-bd0f-8fd66ce6eac1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> 💯 Test performance:\n",
            "Test loss: 1.94\n",
            "Test Accuracy: 41.4%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bT0hnmy2vS26",
        "colab_type": "code",
        "outputId": "ded1ddc8-7863-4444-8415-1b5f7cfbb5f4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Save all results\n",
        "save_train_state(train_state=trainer.train_state, save_dir=config[\"save_dir\"])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> ✅ Training complete!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EvWFdAz0jh4B",
        "colab_type": "text"
      },
      "source": [
        "# Inference"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "15GopFNleSOC",
        "colab_type": "text"
      },
      "source": [
        "Components to see how our trained model fares against unseen surnames."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U7ZUTZbyeW8T",
        "colab_type": "text"
      },
      "source": [
        "### Components"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8t2TD8KJhidm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Inference(object):\n",
        "    def __init__(self, model, vectorizer, device=\"cpu\"):\n",
        "        self.model = model.to(device)\n",
        "        self.vectorizer = vectorizer\n",
        "        self.device = device\n",
        "  \n",
        "    def predict_nationality(self, dataset):\n",
        "        # Batch generator\n",
        "        batch_generator = dataset.generate_batches(\n",
        "            batch_size=len(dataset), shuffle=False, device=self.device)\n",
        "        self.model.eval()\n",
        "        \n",
        "        # Predict\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['surname'], apply_softmax=True)\n",
        "\n",
        "            # Top k nationalities\n",
        "            y_prob, indices = torch.topk(y_pred, k=len(self.vectorizer.nationality_vocab))\n",
        "            probabilities = y_prob.detach().to('cpu').numpy()[0]\n",
        "            indices = indices.detach().to('cpu').numpy()[0]\n",
        "\n",
        "            results = []\n",
        "            for probability, index in zip(probabilities, indices):\n",
        "                nationality = self.vectorizer.nationality_vocab.lookup_index(index)\n",
        "                results.append({'nationality': nationality, 'probability': probability})\n",
        "\n",
        "        return results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7sMDPi4Ieb8f",
        "colab_type": "text"
      },
      "source": [
        "### Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i7TOy8tVVsOA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Load vectorizer\n",
        "with open(config[\"vectorizer_file\"]) as fp:\n",
        "    vectorizer = SurnameVectorizer.from_serializable(json.load(fp))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9zlIp2uJcYHM",
        "colab_type": "code",
        "outputId": "10b1d98d-c776-420f-fc41-e4527d65bd3c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Load the model\n",
        "model = initialize_model(config=config, vectorizer=vectorizer)\n",
        "model.load_state_dict(torch.load(config[\"model_file\"]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "==> 🚀 Initializing model:\n",
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (fc1): Linear(in_features=28, out_features=300, bias=True)\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc2): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fLf7chGfhnFs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Initialize\n",
        "inference = Inference(model=model, vectorizer=vectorizer, device=config[\"device\"])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VChE5EmMgc-S",
        "colab_type": "code",
        "outputId": "7a6bd953-ca26-4798-e711-aa19c02dd805",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "source": [
        "# Inference\n",
        "surname = input(\"Enter a surname to classify: \")\n",
        "nationality = list(vectorizer.nationality_vocab.token_to_idx.keys())[0] # random filler nationality\n",
        "infer_df = pd.DataFrame([[surname, nationality, \"infer\"]], columns=['surname', 'nationality', 'split'])\n",
        "infer_df.surname = infer_df.surname.apply(preprocess_text)\n",
        "infer_dataset = SurnameDataset(df=infer_df, vectorizer=vectorizer, infer=True)\n",
        "results = inference.predict_nationality(dataset=infer_dataset)\n",
        "results"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Enter a surname to classify: Goku\n",
            "<Dataset(split=infer, size=1)>\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'nationality': 'Korean', 'probability': 0.4426434},\n",
              " {'nationality': 'Chinese', 'probability': 0.16157635},\n",
              " {'nationality': 'Japanese', 'probability': 0.15545219},\n",
              " {'nationality': 'Vietnamese', 'probability': 0.12867816},\n",
              " {'nationality': 'Arabic', 'probability': 0.0391046},\n",
              " {'nationality': 'Czech', 'probability': 0.016052954},\n",
              " {'nationality': 'Scottish', 'probability': 0.01287001},\n",
              " {'nationality': 'Polish', 'probability': 0.009812707},\n",
              " {'nationality': 'English', 'probability': 0.008865393},\n",
              " {'nationality': 'German', 'probability': 0.0062941923},\n",
              " {'nationality': 'Russian', 'probability': 0.0054104235},\n",
              " {'nationality': 'Dutch', 'probability': 0.0047801337},\n",
              " {'nationality': 'Greek', 'probability': 0.003674358},\n",
              " {'nationality': 'French', 'probability': 0.0017384152},\n",
              " {'nationality': 'Irish', 'probability': 0.0013247444},\n",
              " {'nationality': 'Spanish', 'probability': 0.0008653855},\n",
              " {'nationality': 'Portuguese', 'probability': 0.00048804158},\n",
              " {'nationality': 'Italian', 'probability': 0.00036844387}]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 72
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5uL1j7a8gJ_h",
        "colab_type": "text"
      },
      "source": [
        "# TODO"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lh2nWwDhgLXD",
        "colab_type": "text"
      },
      "source": [
        "- tqdm notebook"
      ]
    }
  ]
}